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Development of Measures for A Driver Licensing Program 
in the State of North Carolina 
Phase I: Analyses of Current Licensing Tests 



Norman E. Freeberg 
F. Reid Creech 
Educational Testing Service 



INTRODUCTION 



Study Purpose 

The process of evaluating driver capability, for purposes of granting 
or maintaining an operator's license, constitutes a state function that has 
undergone little systematic appraisal of its effectiveness. In most states, 
written tests and driver performance measures are utilized with inadequate 
(if any) knowledge of their value, so that few guidelines are available on 
which to base improvements in the licensing procedures currently applied. 

The purpose of the present study is to examine the measurement 
properties of the driver li ’ -g used i.il the r ' ial.t .*j. north Cl ^ j.ina, 
in order to determine nheir suitability for assessing driver knowledge and 
skill. On the basis of these analyses, it is expected that appropriate 
modifications can be recommended which might eventually enhance the .foc- 
tiveness cx evaluation procedures used by the state in its driver liter ing 
decisions. Since the licensing practices and evaluative procedures cf Forth 
Carolina can be considered reasonably representative of a. number of stares 
in the country, xt is also assumed that the findings would possess sex-: 
degree of relevance to state licensing programs in general. 

Any attempt to define the value of measures used in driver licensing 
and to echie--e meaningful improvements assume, primarily, that: (a) ihe 
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goals or intentions of a driver licensing program can be defined and measured 
adequately, (b) the data needed to arrive at conclusions about existing 
licensing procedures are available from the information system in usable 
form, and (c) there is sufficient research knowledge regarding assessment 
of driver skills to provide a basis for appropriate modificat ions--or com- 
plete revision--of the existing appraisal methods (i.e., the state-of-the- 
research art has advanced sufficiently). The extent to which these broad 
assumptions can be met is not always clear and usually remains unspecified, 
since each implies complex, underlying issues that range from the social, 
legal and administrative to the statistical and methodological.' A number 
of the issues will be touched on over the course of this paper insofar 
they affect analyses of current licensing tests and the implementation of 
recommended improvements. 

The analyses of driver licensing tests to be undertaken here can be 
viewed only as an initial phase of a proposed longer range study program, 
intended to develop a usable assortment of evaluation tools that might yield 
significant improvement over the existing licensing procedures (Freeberg, 
1970). In effect, examination of presently used measures provides a form 
of ’tiase-line " data against which any modifications to those measures (or 
newly designed evaluation techniques) can be compared. Also intrinsic to 
the study is an opportunity to examine the adequacy with which long-term 
performance outcomes, or criteria (e.g., reported accidents and violations), 
can be obtained and utilized. The ability to measure such agreed-upon goals 
of a licensing program accurately represents an important step in demon- 
strating the relative value of any licensing procedures. 
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Goals and Assumptions of Driver Licensing 

Attempts to improve the evaluative tools for driver licensing require, 
at minimum, a degree of common understanding concerning the broad goals or 
intentions of the licensing procedures, since those goals can impose restric- 
tive guidelines on development of more effective driver testing methods and 
on their eventual incorporation in a licensing system. 

Probably the most comprehensive long range intent of driver licensing, 
on -which overall agreement can be reached, is the state's desire to grant 
permission for operation of a motor vehicle to those who demonstrate the 
ability to do so safely and in a manner consistent with the state's legal 
codes (North Carolina State Department of Motor Vehicles, 1970)- Standards 
of good driver performance would, therefore, encompass the ability to drive 
without causing damage and injury to oneself, or others, and without incurring 
citations for violation of existing traffic laws. However, within such a 
broad goal statement, there are several more specific, but unexamined, 
intentions in licensing program practices that require clarification if 
evaluative techniques are to be improved or applied sensibly. One overriding 
and fairly explicit intent is that the driver licensing tests serve a 
selection function and, in effect, screen out those drivers who fail to 
meet basic "minimum requirements for safe driving" (State of North Carolina, 
1970, p. 107). The intent carries with it an assumption that minimum test 
standards can be specified and justified — i.e., that some optimum cutoff 
point (s) can be defined which will (at an acceptable level of probability) 
s rve tv> eliminate those applicants least likely to be "good" drivers and, 
at the same time, license those who would be acceptable. This selection 
model of driver testing has been challenged as to its tenability (i.e.. 
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practicality) and criticized as wasteful of the evaluative potential of 
measures applied to a licensing system in which most applicants are 
eventually granted the license (Miller & Dimling, 1 9 ^ 9 ) « In its place, a 
guidance ox- classification approach has been suggested as more fruitful, 
since the intent would be to define levels of capability for any given driver 
in a number of skill areas = Such information could then be utilized to define 
the educational steps needed to bring the driver to some desired performance 
level. Where appropriate, this same information is seen as useful in guiding 
the imposition of conditions and licensing restrictions on drivers who are 
unable to meet one, or more, of the skill level requirements. 

Under either a selection or classification approach to driver testing, 
however,. it remains essential that the licensing measures bear some degree 
of significant relation to standards of "good" driver performance (i.e„, that 
they be valid). That requirement for test validity carries with it, in turn, 
a second major assumption underlying most programs of evaluation for driver 
licensing purposes which is: that suitable outcome measures or criteria 

reflecting program success (i.e., "good" driver performance) are available 
over the course of the individual's driving career. Recorded violations and 
accidents have invariably served as the long-term outcomes of choice in most 
research efforts. Unfortunately, recorded frequency of accidents and viola- 
tions, as maintained in many state data systems, constitutes less than perfect 
information regarding the true extent of those occurrences (Coppin, McBride, & 
Peck, 1965). For example, many and perhaps most violations go undetected and 
unreported. This is also the case (but probably to a lesser degree) for a 
significant proportion of accidents --particularly those designated as "minor" 
under the state legal codes and in which insufficient damage is inflicted to 
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require an entry in the record system. Furthermore, the precision of these 
criterion measures, as a reflection of driver capability, is weakened by 
inadequate (usually unavailable) information on driver "exposure” and the 
extent to which he is culpable for the accident occurrence. The inability 
to control for such variables, routinely, imposes a severe limitation on any 
licensing program that intends to set meaningful and objective performance 
standards . 

For the most part, problems in defining explicit objectives for the 
licensing program and the relation between the licensing tests and those 
obj ectives have been avoided by tacitly assuming that the test scores possess 
"face validity." That is, the test is assumed to be an adequate representa- 
tion of one r s knowledge of the content of the staue r s Driver Handbook in and of 
itself. The reasoning is acceptable only if there is no further implication 
that a higher score on the written test reflects superior driver capability 
in any way. On a similar basis, better rated performance (i.e., a higher 
score )on a conventionally administered road test can be assumed to define 
"superior" driving skill, but this still remains only face valid until one 
demonstrates the relationship (predictive value) of that measure to later 
driving competence. Although reliance on face valid measures may stem from 
practical necessity in the conduct of a licensing program, justification of 
the tests solely on the basis of that assumption tends to stifle any attempts 
to improve currently used driver evaluation techniques. It is also In 
obvious contradiction to formal, stated, intentions of driver licensing 
laws which deal specifically with concepts of posttest "safety" and driving 

"ability" as goals (State of North Carolina, 1970). 

The third relatively unexamined and untested major assumption of most 

driver licensing programs (and implicitly related to the two previous ones) 
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is that the system possesses a built-in, monitoring function capable of: 

(a) detecting "flaws" in driver performance on the basis of available 
information and (b) instituting appropriate corrective action intended 
either to restore suitable driver performance or to revoke the driving 
privilege . Detection of lapses in capability for licensed drivers is 
usually dependent on information derived from periodic license renewal 
examinations (written tests , road tests , eye tests , etc.), along with in- 
formation about the frequency and nature of the driver's accident and 
violation involvement (e.g., usually based on some form of "point system"). 

Here again, however, one is essentially in the same position as with initial 
licensing evaluation, in being obliged to demonstrate, empirically, the 
effectiveness of such procedures when they are used to make licensing 
decisions. The problem faced in dealing with licensed drivers is to define 
levels of ineptitude that warrant given levels, or forms, of corrective 
action (i.e.. How many violations, accidents or accumulated points warrant 
license withdrawal? For how long? When is a suitable level of driver 
capability restored? ) . 

Evidence that bears on the acceptability of any of the above major in- 
tentions or assumptions, underlying state licensing procediires, is exceedingly 
scarce. What little direct evidence is available can not only be considered as 
incomplete, but also as failing to provide strong support for the effective- 
ness of c onveii ci onal driver evaluation techniques applied during any phase 
of the licensing process. With regard to written tests of driver knowledge 
used for granting an initial license, for example, the only reported analyses 
have dealt with small samples (i.e., approximately 100 drivers) from two 
states (Michigan and Connecticut) and indicated little in the way or pre- 
dictive validity against criterion ratings of driver proficiency (Eno 
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Poundation, 1948). Similarly, a test composed of driving maneuvers, derived 
from those used by a number of states in their initial licensing road 
tests, produced negligible validity when applied to groups of drivers 
classed as "accident ,' 1 ’Violation- free " and "problem" drivers (McGlade, 1963). 
Analyses of the- road test scores, used for initial licensing by North 
Carolina, also failed to demonstrate significant discriminability between 
drivers who had incurred accidents and those who were accident and violation 
free. However, individual items (maneuvers) for certain classes of driving 
skill (e.g., "physical handling of the automobile," "interaction with traffic") 
showed some discriminative value, but at levels that the investigator felt 
were minimal for practical predictive applications (McCrae, 1968). 

In the monitoring of driving performance for persons already licensed, 
no efforts have been reported that deal with analyses or validation of license 
renewal examinations and only one attempt has been made to validate an oper- 
ational accident-violation point system (Coppin & Peck, 1967 ) . Evidence 
that is largely indirect would appear to substantiate some limited value for 
a monitoring system that incorporates accident and violation occurrences 
from driver history, in combination with such personal demographic character- 
istics as age, education, marital status, years of driving experience, miles 
driven, etc. (Coppin, McBride, & Peck, 1967 ; Goldstein, 1961, 1964). Unfortu- 
nately, as with other evaluative techniques, the practical value of the driver 
record and background information for predicting subsequent accidents and 
violations has been found to be severely limited- -although considered by some 
invest igai. ~>rs "encouraging" and worth future attempts to identify drivers 
who are likely to be greater safety risks (Coppin, McBride, & Peck, 1967 ) . 

.Evidence bearing on the suitability of driver performance measures that 
can serve as licensing program outcomes (criteria) has also been sparse. The 




10 



- 8 - 



need is to demonstrate which criterion variables and scoring methods provide 
the most defensible standards for validating licensing decisions or written 
tests. Recorded violations and accident occurrences , and associated forms 
of information (e„g., injuries and fatalities), have been--and are likely to 
continue--as the "ultimate" program standards required for administrative 
decisions. Too often, however, these must be accepted in whatever form they 
are recorded for the state's licensing system. Data organization and formats 
are often designed for retrieval and applications best suited to administra- 
tive purposes, rather than for research or study needs. Coupled with 
problems of format and accessibility- is the suspected lack of precision 
for criterion measures based solely on accident -violation occurrences 
(i.e., as previously mentioned, they often fail to represent the total 
accident and violation picture, as well as fail to take into account 
exposure and accident culpability for each driver) . In addition, it should 
be mentioned, briefly, that accidents and violations are particularly difficult 
criterion variables to analyze by conventional statistical techniques because 
of the relative rarity of their recorded occurrences in the population, and 
the corresponding extreme skewness of their distributions (i.e., Poisson in 
shape). The effect, especially for accidents, is one that makes logical 
interpretation of results difficult, since it diminishes the reliability of 
any accident (or accident -dependent ) measure, and severely restricts the 
ability to predict that criterion variable (Coppin, McBride, & Peck, 

1965). Further increases in precision of measurement for accident- 
violation scores would appear dependent on additional, meaningful, forms 
of information about the characteristics of those occurrences in terms 
of the driver and the driving situation (e.g., driver exposure, severity 
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and nature of accidents, culpability, type and condition of automobile 
driven, etc.). 

Part of the solution to the criterion problem has also been sought in 
the development of more readily available, standardized, driving-criterion 
tasks likely to be indicative of longer term driver capability. Here 
the attempt has been to define driver skills on the basis of a wide variety 
of "realistic” and carefully measured performance indices (Greenshields & 

Platt, 1969 ) that possess demo ns tr a; - relationships to driver accident and 
violation histc" r . Development of more * mediately available and more 
accurately measurable driver performance criteria, to supplement the longer 
term accident -violation measures , has been considered an important aspect 
of any future test validation and research efforts. 

The Driver Licensing Process 

As indicated previously, this study will be based on the driver licensing 
system of the State of North Carolina. In order to -understand the study in- 
tent, its scope and the results presented, it is necessary to have a general 
understanding of the licensing procedures and requirements imposed by that 
state . 

Typically, the applicant who desires a license to operate a private 
passenger vehicle, for the first time, must pass examinations that evaluate 
his: (l) visual acuity; (2) ability to identify, by color and shape, the 

standard traffic signs used on streets and highways; ( 3 ) knowledge of 
"traffic laws and safe driving practices" as determined by a 30-item Rules 
Test, available in either written or oral form; and ( 4 ) "actual driving ability” 
by means of a road test that is administered only to applicants who have 
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achieved a passing score on the first three examinations (North Carolina, 

Dept . of Motor Vehicles, 1969)= In the event of failure to pass any of these 
licensing requirements the applicant may repeat them when he believes that 
he is ready to do so — although it is s^-gested in the Driver's Handbook that 
one week be allowed to lapse before ree>. mi .^:on 9 

All operators' licenses are valid fc fc :r ye s from the year of 
issuance, at which time the licensee must n ’ ego o renewal licensing 
examination (written or oral). This test is _mila .r in format and content 
to the initial licensing rules test, but ccutt is or..; r 20 test items. (The 
renewal license applicant may, under unusual :.rcur.~ances , be required to 
take a road test at the discretion of the ex:-.. — mer ; but such a requirement 
is highly atypical •) 

Continuity in the monitoring of driver ability and performance is 
achieved not only through the formal testing procedures but also through a 
penalty system of "points” assigned on the basis of convictions incurred 
for various minor traffic violations 0 A warning letter is sent to the 
licensee after four violation points are accumulated and another after the 
accumulation of seven points * License suspension is possible after 12 such 
penalty points have been assigned. Mandatory suspensions and revocations 
are also imposed for single violation occurrences of greater severity (e.g* , 
driving while intoxicated, manslaughter, hit and run violation). 



■x- 

There are no formally prescribed limits set with regard to the time 
between examination and reexamination^ or with regard to the number of 
times the applicant may be reexamined., 
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Scope of the Study 

This report will focus exclusively on the written rules tests used for 
initial award of the driver T s license and for license renewal. : suita- 

bility of the measures is to be det ermined, largely , on the basis o* their 
internal measurement characteristics. Included in that determine,- m will be 
the examination of item means (i.e., difficulty levels), variances; corre- 
lations of items with the total test score., the reliability of the total 
test, the underlying dimensions (or factors )that define the test content 
and? on the basis of those results, the overall comparability of various 
test forms that are administered interchangeably. 

In addition, and perhaps most important, are analyses of the rules test 
capability to discriminate between "good" and "poor” drivers in terms of the 
accidents and violations that they have incurred (i.e., test validity) • This 
analysis is possible in the present study for the written license renewal rules 
test forms only. By utilizing the recorded accident and violation occurrences 
(covering the four-year period prior to completion of the written renewal test) 
for a sample of renewal- license applicants, it is possible to determine the rela- 
tionship between those occurrences and the test scores. Such a post - hoc , 
or retrospective, study is somewhat less than ideal, since it entails an after- 
the-fact examination of the previous four- year driving performance record 
for those drivers who have remained in the state and retained their licenses 
(i.e., there is the likelihood of an unknown degree of selective bias). More 
desirable and more definitive would be a predictive, or prospective, validation 
in which the written rules test scores are correlated with posttest driver 



performance. The data necessary to accomplish that form of validation were 
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not available at the inception of this study, although they are in the 

“X* 

process of being obtained. 

As an adjunct to analyses of the licensing tests, it is of value to 
consider whether scores on these measures, and subsequent driver 
performance, are influenced by different background characteristics of the 
applicants. Evidence to support the potential influence of driver charac- 
teristics or experiences on accident and violation involvement has been fairly 
extensive (Coppin, McBride, & Peck, 1967, Goldstein, 196k; McFarland & Moseley, 
195k) . Therefore, a small number of available background variables obtained 
from a driver personal data form were analyzed, in combination with the rules 
test scores, to determine the degree to which they might enhance the relation- 
ships between the written test scores and the accident-violation occurrences 
obtained from the driver history. 



METHOD 

Description of the Measures 

Features of the licensing rules tests, accident and violation criteria 
and background data pertinent to the present study are as follows : 

1. Initial Licensing Rules Test (Written form) — is a 30-item, untimed, 
paper and pencil measure available in five forms, each of which is similar 
in content and format. The test contains 20 items in four— choice, multiple- 
choice format and 10 items in a true-false format. Item subject matter is 



Initial licensing rules test scores for renewal license applicants are 
not retained in the data system, so that a predictive (prospective) study was 
not possible with that group. However, determination of the predictive 
validity of the initial licensing rules tests is planned as part of a supple- 
ment to the present report, based on one year posttest accidents and 
violations for a sample of applicants tested in May 19 ^ 9 - 
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drawn solely from a handbook on "Traffic Law and Highway Safety" (Stat if 
North Carolina, '1970 ) which contains material describing the state's Tic 

rules and regulations* driving techniques and emergency procedures. A. 
test items are given equal weight in the total score and a passing scor 
is defined as 70 $ or more of the items answered correctly. (Test secui ^y 
prevents inclusion of any of the initial licensing test forms. Howeve. some 
indication of the content of each item is presented in the tables of tn 
Results and Discussion section.) 

2. Renewal Licensing Rules Test (Written form) --is a 20 -item meas r - 
in a four-choice multiple -choice format* with five available test forms 
each form containing item material based on the "Driver T s Refresher Har. ,ook 
of Traffic Laws and Highway Safety" (State of North Carolina* 1969)* That 
volume summarizes much of the material in the previously cited handbook used 
by initial license applicants. Each item is weighted equally and lh or more 
items answered correctly (i.e.* 70$) constitutes a passing score. An obsolete 
form of the renewal rules test can be seen in Appendix A. It should be noted 
that those items are identical in format and similar in content to those of 
the currently used renewal rul.es test forms and equally similar to the 20 

-X- 

multiple- choice items used in the initial licensing rules test. 

5 . The Supplementary Data Form - -contains some categories of information 
that are not customarily required as part of the formal renewal licensing 
procedure. It was administered* voluntarily* to a sample of renewal licence 



*An oral;, untimed test is also 'available in five forms for those appli- 
cants unable to read well enough to complete the written rules tests. The 
same five forms are used for initial and renewal licensing. However, the 
oral test is not logically amenable to detailed analyses as presently adminis- 
tered, since the examiner halts the presentation when the applicant fails more 
than 30 $ of the items. Interpretation of item characteristics for am untimed, 
uncompleted measure is considered to be of sufficiently quest ionabl-: value 
to negate any detailed analyses of its measurement quality. 
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applicants at the time of renewal rules test administration. The form 
presented in Appendix D is designed to obtain--in as short a space as 
possible--some of the background information for which significant validity 
against accident -violation criteria had been claimed in previous studies. 

The information provided,, that was found useful for analysis, includes 
education, sex, marital status, driving experience (years of driving), age 
of car and applicant's estimates of miles driven per week. 

4. Recorded Accidents and Violations (Criteria) --were available from 
driver records that are maintained as part of the state licensing data 
system. The reported violations represent only those occurrences for which 
convictions were obtained; while the frequency of accident involvement 
represents those instances reported and confirmed by police accident reports 
Recorded accidents are defined as those in which any person is killed, or 
injured, or $100 or more of property damage is incurred. 

The total frequency of accidents and violations sustained over the four, 
year period, preceding administration of the written license renewal rules 
test, constituted the two criterion variables for renewal test validation 
purposes. The frequency scores for each variable were intended for use 
separately and in optimally weighted combination. However, the reader 
should be aware that the two measures are not completely independent and 
possess a degree of "spurious " overlap when they are used as combined 
criteria. Thus, although accident and violation frequency are correlated 
with each other in the present samples at a moderate level (r = .4o),some 
undefined portion of that relationship results from the fact that accidents 
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can be accompanied by a violation conviction for one or more of the drivers 
involved. 



Study Samples 

The samples consisted of 21,671 male and female applicants who completed 

the Initial Licensing Rules Test in Spring of 1969 and 8,187 male and female 

renewal license applicants who took the renewal rules test in January of 

1970* Applicants were drawn from test centers throughout the state of 

North Carolina. Although the scores of those applicants who fail the tests 

are not normally retained in the data system, the procedure was changed 

for present study purposes so that all rules test scores were made available 

(i.e., the total range of scores for those who passed oh failed the tests 

** 

appears in this sample ) . 

The sample sizes for each of the five test forms of each rules test are 
about evenly divided over the total sample. That is , about 1,600 respondents 
comprise the sample that completed each form of the renewal licensing rules 
test and about 4,500 who completed each form of the written initial licens- 
ing examination. Exact sample sizes for each form are presented in the tabled 
results of the next section. 

For the validation phase of the renewal rules test analyses, the sample 
was composed only of those drivers for whom complete data were available in 



It was not feasible with the present data system to remove accident-free 
violations from the total number of violation occurrences for any given driver. 

It should be understood, however, that for either of the rules tests an 
undetermined number of applicants may have failed to pass the tests on one or more 
occasions prior to this administration. There is no way of.identifying such 
individuals ,or the number of previous test administrations jfrom the existing 
data system. 
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the form of the rules test score, background information and driver history. 
That sample totaled 4,4-74 males and females. Specific subsample sizes, by 
rules test form and sex, are appropriately indicated in the tabled results. 

Data Analyses 

Two major- types of analyses are to be undertaken in order to determine 
the measurement characteristics and value of the various forms of the written 
renewal examinations. One involves analysis of rules test characteristics 
at both item and total scale levels. The other entails the validation of 
the renewal rules test, using accident and violation data as the driver per- 
formance criteria. As part of the latter analysis, the background variables 
are to be incorporated as predictors, along with the renewal rules test 
scores, in optimally weighted combination. 

Specifically, the analyses are to deal with: 

(1) Rules Test Characteristics --determined on the basis of (a) total 
scale reliabilities — as alpha coefficients, (bj item means or 
difficulty levels --as proportion of respondents passing each item, 
(c) spread of item scores — as item variance, (d) item "compat- 
ibility" with the total scale--as item, total -test score correlations 
and (e) underlying dimensions that define coherent item groupings 

or potential subscales — as factors obtained from a principal com- 
ponents analysis and a varimax rotation to orthogonality (Kaiser, 

195 ^). 

(2) Rules Test Validation - -obtained from correlations between each of 
the written renewal rules test forms and the ac ident and violation 
occurrences recorded for each driver over the previous four -year 
period. The validity coefficients were obtained for accidents and 

19 
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violations separately and for both variables in optimally weighted 
combination. Scores for the accident and violation criteria were 
derived from a square-root transformation of the accident and 
violation frequencies in an attempt to convert the highly skewed 
distributions to as nearly normal shapes as possible. (Because 
of the nature of these Poisson distributions, however, there was 
relatively minor benefit derived from that transformation. ) 

All correlations for the validation were computed separately 
for male- and female samples. 

( 3 ) Backgro und Correl ates — of driver accident and violation performance 
were obtained from the relationships between those criteria and the 
variables from the Supplementary Information Form. The correlations 
were computed between each of five background variables along with 
each renewal rules test form and each of the criteria (as zero-order 
r's). Optimally weighted combinations of those same predictors, with 
the optimally weighted composite of accidents and violations, were 
also correlated with one another (as canonical R's). These corre- 
lations were also computed for male and female samples separately. 

RESULTS AND DISCUSSION 

Initial Licensing Rules Test (Five Written Forms) 

The measurement characteristics of the five forms of the initial licens- 
ing tests are shown in Tables 1 through 5 in terms of item difficulty levels 
(means), item variances, item-total test score correlations and total scale 
reliability estimates (alpha coefficients). In Tables 6 through 10, factor 
loadings are presented for those same test forms on five rotated varimax 

er|c £o< . 
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factors o Furths:; detailed information concerning the item characteristics 
has been reserved for Appendix B in terms of the proportion of times each 
item alternative was chosen. Such information serves to pinpoint the bases 
for the other results obtained and also serves as a major source of informa- 
tion in any attempt to modify the existing items . 

From the results obtained, the following interpretations are possible: 

1. It em Characteristics : The item means, or difficulty levels , shown in 

the first column of Tables 1 through 5 , form a distinctly skewed distribution 
with items tending to be overly easy. The values indicate a relatively high 
probability that most applicants will answer any given item correctly. 
Specifically, on each of the five test forms it can be seen that the range 
of p values is in excess of »80 for as many as 8 of the 30 items in 
Form ^1 to as many as 12 of the 30 items in Form # 5. Correspondingly, 
there are relatively few items below the .50, or moderate difficulty level. 
The true-false items (21-30) account, proportionally, for the largest 
number of excessively easy items and none of the moderately difficult items. 
Some attempts at more optimum item "peaking" (Guilford, 195^0 is clearly in 
order and it is apparent that the items in the T-F format are detrimental 
to the quality of the tests in this respect. 

Item variability (presented in the second column of each table) is 
dependent directly on the item mean scores, such that variance is reduced to 
minimal levels for items having extreme values for their difficulty indices. 
Such minimal amounts of item variance tend to reduce the likelihood of find- 
ing useful overall test variance. Thus, the items with means falling between 
.30 and .70 can be said to show the most acceptable amounts of variance for 




inclusion in these measures. 
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Table 1 

Initial Licensing Rules Test (Form #l) 

Item Characteristics and Reliability Estimates 

(N = 4352) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1. 


Brake & clutch operation 


.48 


.25 


• 36 


2. 


Braking distance 


• 52 


.25 


.44 


3 . 


Speed regulations 


.65 


.23 


• 38 


4 . 


Use of lights 


.35 


.23 


• 25 


5 . 


Accident statistics 


.4i 


.24 


• 33 


6. 


Safe following distance 


.68 


.22 


• 43 


7- 


Pedestrian regulations 


•70 


.21 


• 38 


8. 


Hand signals 


.91 


.08 


• 37 


9. 


Traffic signs 


.66 


.23 


.38 


10 . 


Accident statistics 


.49 


• 25 


.28 


ll. 


Intersection regulations 


.81 


• 15 


• 37 


12 . 


Traffic circle regulations 


• 53 


• 25 


.46 


13. 


Accident statistics 


.76 


• 19 


.46 


l4. 


School bus laws 


.85 


.13 


.4i 


15. 


Time allowance for trips 


• 53 


• 25 


• 39 


16 . 


"Safety-Responsibility Law" 


.63 


• 23 


• 39 


17. 


Accident statistics 


• 53 


• 25 


• 39 


18 . 


Licensing regulations 


.81 


• l6 


.40 


19. 


Licensing rules 


•70 


.21 


• 35 


20 . 


Accident statistics 


•70 


.21 


• 32 


21. 


Signal regulations' 


.86 


.12 


• 27 


22. 


Passing regulations 


.60 


.24 


• 31 


23. 


Reaction time 


.82 


• 15 


• 34 


24. 


Automotive facts 


•75 


• 19 


• 31 


25. 


Accident statistics 


.62 


.24 


• 37 


26 . 


Speed limits 


.69 


.22 


.40 


27. 


Intersection rules 


.86 


.12 


• 38 


28 . 


Use of lights 


.63 


.23 


.23 


29. 


Safe driving habits 


.61 


.24 


.40 


30. 


Intersection regulations 


• 91 


.08 


.29 



Coefficient alpha = .74 

Total Test Mean = 20.0 
Total Test SD = 4.8 
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Table 2 

Initial Licensing Rules Test (Form #2) 

Item Characteristics and Reliability Estimates 

(N = 4360) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1. 


Speed regulations 


°37 


.23 


.23 


2, 


Safe driver attitude 


•79 


.17 


.46 


3 . 


Control of car 


.58 


.24 


• 55 


4 . 


Traffic laws (road safety) 


.88 


.11 


.42 


5 . 


Reaction time 


.66 


.22 


.37 


6. 


Railroad crossing regulations 


.56 


.25 


.40 


7. 


Emergency traffic control 


.83 


.14 


.51 


8. 


Passing regulations 


.84 


.13 


.35 


9. 


Clover-leaf regulations 


.61 


.24 


.48 


10. 


Accident statistics 


*57 


.25 


• 37 


11. 


Passing regulations 


.58 


.24 


.35 


12. 


Response to emerge veto signals 


.85 


.13 


.23 


13. 


Time allowance for trips 


.58 


.24 


.35 


l4. 


Accident statistics 


.68 


.22 


.34 


15. 


Car maintenance 


.87 


.11 


.45 


16 . 


Speed regulations 


.56 


.25 


.31 


17. 


Traffic circle regulations 


.63 


.23 . 


.42 


18 . 


Accident statistics 


.94 


.06 


.41 


19. 


Rate of speed 


.34 


.22 


.32 


20. 


Accident statistics 


.50 


• 25 


.33 


21. 


Causes of skidding 


.84 • 


.14 


.30 


22. 


Hand signals 


.70 


.21 


.28 


23. 


Passing regulations 


.60 


.24 


.30 


24. 


Life of tire 


.67 


.22 


.29 


25. 


Road signs 


.67 


.22 


.43 


26 . 


Good driving habits 


.78 


.17 


.46 


27. 


Braking - road conditions 


.80 


.16 


.41 


28 . 


Accident statistics 


.57 


.24 


.37 


29. 


Use of lights 


.63 


.23 


.20 


30. 


Dangers of drinking 


.88 


.11 


.36 



Coefficient alpha = .75 

Total Test Mean =20.3 
Total Test SD = 4.8 
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Table 5 

Initial Licensing Rules Test (Form #3) 

Item Characteristics and Reliability Est im ates 

(N = 4207) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1 . 


Use of signals 


• 71 


.21 


• 32 


2. 


Use of signals 


.64 


.23 


.33 


5. 


Accident statistics 


.68 


.22 


.33 


4. 


Operating procedures 


.69 


.21 


.46 


5- 


Reaction time 


.67 


.22 


.32 


6. 


Safe night driving 


.67 


.22 


.44 


7- 


Intersection regulations 


.62 


.24 


.42 


8. 


Accident reporting 


.81 


.15 


• 39 


9. 


"Safety Responsibility Law" 


.68 


.22 


.41 


10. 


Safe driving habits 


• 93 


.06 


.42 


li. 


Accident statistics 


.65 


.23 


• 45 


12. 


Operating procedures 


•70 


.21 


.41 


13. 


Intersection regulations 


•79 


•17 


.43 


14. 


Licensing regulations 


.86 


.12 


.38 


15- 


Road markings 


.82 


.15 


.43 


16. 


Braking distance 


.50 


.25 


.42 


17. 


Parking rules 


.62 


.24 


.34 


18. 


Permit regulations 


.89 


.10 


• 35 


19. 


Braking laws 


.43 


• 25 


.30 


20. 


Safe driving habits 


.82 


.15 


.47 


21. 


Passing regulations 


.60 


.24 


.31 


22. 


Intersection regulations 


.84 


.13 


.22 


23. 


Hand signals 


.69 


.21 


.29 


24. 


Reaction time 


.80 


.16 


•39 


25. 


Automotive facts 


•75 


• 19 


• 30 


26. 


Causes of skidding 


.84 


.14 


• 34 


27. 


Road signs 


.69 


.21 


•45 


28. 


Intersection regulations 


.91 


.08 


.34 


29. 


Safe driving - slippery pavement 


.84 


.14 


.40 


30. 


Life of tire 


.67 


.22 


• 27 



Coefficient alpha = .75 

Total Test Mean = 21.8 
Total Test SD =4.7 
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Table 4 

Initial Licensing Rules Test (Form #4) 

Item Characteristics and Reliability Estimates 

(N = 4351) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1. 


School bus law 


• 97 


.02 


.23 


2. 


Driving hazards 


.74 


.19 


36 


3 . 


License suspension rules 


.65 


23 


.26 


4 . 


Licensing rules 


.85 


..13 


.34 


5 . 


Safe night driving 


.68 


,22 


.43 


6. 


Passing capability 


• 32 


22 


. 33 


7- 


Changing lanes 


.87 


.11 


.28 


8. 


Traffic lights 


.80 


16 


• 39 


9. 


Road markings 


.87 


. _2 


.43 


10 . 


Intersection regulations 


.44 


.25 


■ 43 


11. 


Parking rules 


.60 


.24 


= 33 


12 . 


Expressway safe speed 


.37 


.23 


.43 


13. 


Pedestrian regulations 


•74 


.19 


.40 


i4. 


Road, conditions 


.76 


.18 


.38 


15. 


Accident statistics 


.65 


.23 


.45 


16 . 


Operating procedures 


.70 


.21 


.49 


IT- 


Reaction time 


. 66 


.22 


• 35 


18 . 


Headlight regulations 


.36 


.23 


.23 


19. 


Unobstructed vision 


• 91 


.08 


.36 


20 . 


Speed regulations 


.4i 


.24 


.26 


21 . 


Safe driving habits 


.61 


.24 


.36 


22 . 


Causes of skidding 


.85 


.15 


.30 


23. 


Intersection regulations 


.86 


.12 


.25 


24. 


Intersection regulations 


.91 


.08 


• 33 


25. 


Speed limits 


.69 


.21 


.38 


26 . 


Automotive facts 


•75 


• 19 


.36 


27. 


Dangers of drinking 


.91 


.09 


• 37 


28 . 


Life of tire 


.66 


.22 


.27 


29. 


Signal regulations 


.88 


.11 


.27 


30. 


Reaction time 


.84 


.14 


• 34 





Total Test Mean = 21.3 
Total Test SD =4.3 



Coefficient alpha = .72 
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Table 5 

Initial Licensing Rules Test (Form jj= 5 ) 
Item Characteristics and Reliability Estimates 

(N = 4401) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1 . 


Traffic laws 


.88 


.11 


.41 


2. 


Permit regulations 


• 92 


.08 


.35 


3- 


Reaction time 


.68 


.22 


.52 


4. 


Speed regulations 


.63 


.23 


.38 




Signal usage 


• 75 


.20 


.37 


6., 


Road markings 


.91 


.08 


.43 


7 - 


Road markings 


.64 


.23 


.45 


8. 


Car maintenance 


.88 


.11 


.50 


9. 


Intersection regulations 


.47 


.25 


38 


10. 


Railroad crossing safety 


.83 


.14 


.43 


11 . 


Accident statistics 


.50 


.25 


.33 


12. 


Accident statistics 


.78 


.17 


.31 


13 o 


Use of lights 


.70 


.21 


.31 - 


l4. 


Dangerous driving 


.85 


.13 


.49 


15- 


Car maintenance 


.43 


.25 


.34 


16. 


Speed requirements 


.75 


.19 


.40 


17- 


Accident statistics 


.67 


.22 


.33 


l8. 


License suspension rules 


.45 


.25 


.32 


19. 


Changing lanes 


.89 


.10 


• 32 


20. 


Use of lights 


.63 


.23 


.37 


21. 


Road signs 


.68 


.22 


.45 


22. 


Life of tires 


.66 


.22 


.26 


23. 


Carbon monoxide danger 


.84 


.13 


.21 


24. 


Good driving habits 


.81 


.15 


.45 


25. 


Intersection regulations 


.87 


.11 


.41 


26. 


Signal regulations 


.87 


.11 


.29 


27. 


Dangers of drinking 


.91 


.08 


.37 


28. 


Safe driving - slippery pavement 


.78 


•17 


.37 


29. 


Use of lights 


.61 


.24 


.29 


30. 


Safe driving habits 


.59 


.24 


.40 



Coefficient alpha = .74 

Total Test Mean = 21.8 
Total Test SD = 4.5 
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The item, total-test score correlations (third column of each table) 
simply reflect the degree to which each item measures something similar ~o ' 
the scale of which it is a part--i.e., the extent to which it "belongs" . n 
the scale and its contribution to scale reliability. The values shown here, 
for this particular statistic, are fairly substantial (i.e., correlations 
largely in the .30' and .40 ’s) and also .ndicate that the test forms are 
likely to show a re sonable degree of overall reliability or internal con- 
sistency. 

A brief look tz selection proportions: for the items of each test m 
(Appendix B), sufi ; ;es to inaicace which items incorporate distractors having 
such low proba.bil.03r of selection that they are virtually sueless for t_.. ir 
intended purpose (e.g. , those with p values of less than .03). 

2. Form Reliability : Alpha coefficients, shown at the bottom of each 

table, represent lower -boundary (conservative) estimates of internal con- 
sistency for each test form. These are uniformly high and similar from form 
to form (r's in the low to middle ,70 's), which is consistent with the pattern 
of item-total test correlations discussed above. Such levels of reliability 
are considered acceptable for a paper and pencil test, but represent only 
one necessary index of scale suitability and, certainly, not the primary 
one on which to base any decisions regarding the value of these initial 
licensing test forms. 

3* Test Factors (Empirical Item Groupings): Loadings for the five 

factors extracted from each form, and rotated to orthogonality, are shown in 
Tables 6 to 10 , along with the proportion of total test variance accounted 
for by each factor. Based on factor variances and the number of loadings of 
interpretable magnitude (i.e., .30 or greater), five factors were found to be 
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Table 6 

Initial licensing Rules Test (Form #l) 
Rotated Varimax Factor Loadings 



Item 


Factor 1 


F'actor 2 


F'actor 3 


Factor 4 


Fi. otor 5 


1 


.04 


C O 
* — 4 


-.01 


.24 


-.00 


a 


• 24 


.81 


-.05 


.16 


35 


** 


. 81 


• 37 


.04 


.05 


,20 


4 


-.10 


• 42 


.01 


-.03 


09 


5 


.03 


.06 


-.01 


.59 


17 


6 


• 2i 


.17 


.34 


.18 


26 


7 


. 20 


.43 


.12 


.05 


00 


8 


.27 


.SI 


. 42 


-.03 


-.04 


9 


.15 


. 44 


.19 


.07 


-.07 


10 


.02 


.09 


-.02 


• 59 


-.12 


11 


.31 


,20 


.22' 


.02 


-.00 






• 30 


.05 


-3 4 


. c / 


13 


'.38 


.26 


.20 


.01 


.17 


14 


.'2SL 


.19 


.32 


.10 


-.20 


15 


.16 


.15 


.01 


.37 


. 84 


16 


.34 


.14 


.17 


.08 


-.00 


17 


.43 


.07 


■ .00 • 


.03 


.16 


18 


•37 


.01 


.20 


.27 


-.04 


19 


.09 


.15 


. 24 


.21 


.3.8 


80 


. 16 


-.04 


.21 


.41 


-,C9 


81 


-.04 


.07 


• 50 


.07 


.09 


88 


.19 


.11 


-.02 


L- —.05 


.52 


83 


.39 


.01 


.22 


-.03 


.05 


84 


.01 


-.14 


.41 


.35 


. 1 p 


85 


. 51 


-.11 


-.07 


.09 


.26 


86 


.4 6 


. 22 


.09 


-.02 


-.00 


r, r ~ 
to ( 


. 87 


.06 


• 42 


.06 


-.05 


23 


— * is 


-.11 


.27 


.07 


* 66 


89 

30 


.55 

.05 


.01 
. 06 


-.05 

.57 


.16 

-.03 


-.01 

.07 




Percent cf 


variance (from 


Principal 


Components) 






13.6% 


4 2 


4.0% 


A ^ o« 

3 « O/y 


5.6% 



0 2S 

ERIC 



-26- 



Table 7 

: ait^al Licensing Rules Test (Form #2) 
Rotated Varimax Factor Loadings 



Item 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


1 


— • 0 ^ 


-.02 


.17 


-.00 


.62 


2 


.47 


.10 


.14 


.11 


.18 


3 


.43 


.24 


• 36 


-.18 




4 


.47 


.06 


.13 


.29 


02 


5 


.18 


.07 


.41 


.05 


.09 


6 


.10 




.29 


.06 


.08 


7 




.18 


.18 


.02 


.09 


8 


.22 


.18 


.11 


.46 


.00 


9 


.22 


.43 


.33 


.00 


-.06 


10 


.23 


.40 


.00 


-.04 


.07 ■ 


11 


.17 


-.01 


.50 


.03 


-.07 


12 


.12 


.05 


-.03 




.02 


“1 o 


0?-. 


7 5* 


45 




7Sk 


14 


.16 


. 0.4 


. 80 


. 24 • 


.32 


15 


.49 


.00 


.22 


.19 


.10 


16 


.17 


.06 


-.01 


-.06 


. 65 


17 


.18 


.41 


.35 


-.06 


-.19 


18 


.52 


-.03 


.12 


.39 


.01 


19 


-.05 


.13 


.52 


-.00 


.18 


20 


.08 


•25. 


.06 


.05 


,16 


21 


.42 


-.03 


.03 


.12 


.07 




.07 


.37 


.07 


.08 


-.09 


23 


.00 


.43 


.12 


.05 


-.06 


% 

£•*1 


.03 


• 25 


.16 


.05 


.17 


25 


.43 


. 28 


.04 


-.20 


.08 


26 


.59' 


.07 


.10 


-.02 


.03 


27 


.55 


. 22 


.05 


-.02 


.02 


28 


.3^ 


.34 


-.14 


-.26 


.15 


29 


-.15 


.59 


-.26 


.29 


.17 


30 


.50 


.09 


-.08 


.12 


.08 




Percent of variance (from Principal 


Components) 






14 . 8% 


4.8% 


3.8% 


3.7% 


3.5% 




23 



1 

2 

3 

l 

*+ 

5 

6 

7 

3 

9 

10 

11 

12 

n? 

~\ >. 

— ►»• 

15 

16 

17 

18 

19 

20 

21 

22 

S3 

84 

25 

26 

2? 

28 

29 

30 
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Table 8 

initial Licensing Rules Test (Form #3) 
Rotated Varimax Factor Loadings 



‘’acts • .1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


.01 


-.00 


.07 


. 66 


.10 


.1 1 


-.03 


. 20 


.41 


-.05 


. 2 ; 


.24 


-.09 


.34 


-.13 


. 2i 


.44 


.10 


.IS 


.02 


.0= 




-.13 


.10 


-.05 


.21 


.13 


.07 


.50 


-.00 


.34 


.28 


.20 


.04 


-.09 


.21 


.09 


.26 


.15 


.26 


.16 


.39 


.11 


.09 


.19 


.58 


.11 


-.04 


.09 


.19 


.20 




.31 


.07 


.07 


.21 


. 14 


.34 


-.04 


-.05 


<45 


.17 


.m 


.15 


.00 


.m 


.07 


. 01 


.03- 


._L'7 


.51 


.16 


.18 


.02 


-.08 


rsr 


.22 


.47 


.18 


-.02 


.23 




. 20 


-.20 


-.07 


.50 


.00 


-.04 


.01 


.30 


.04 


-.00 


. 60 


-.03 


-.03 


• 51 


.14 


.06 


.22 


.06 


-.02 


.05 


.40 


.17 


.10 


.03 


.02 


-. 03 


.04 


•64- 


-.10 


.00 


.31. 


. 40 


.15 


.14 


.44 


.18 


' -.01 


.10 


.01 


.18 


-.06 




.39 


• 36 


-.08 


.26 


-.06 


.29 


.29 


.11 


.33 


.26 


-. 01 


.29 


.08 


.11 


-.09 


.49 


.19 


.82 


.27 


.08 


.27 


-.27 


.54 


.10 


.05 


, 27 



Percent of variance (from Principal Components) 
K ■ 4.6% 3.9% 3.6% 



O VO/ 



30 
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Table 9 

Initial Licensing Rules Test (Form #4) 
Rotated Varimax Factor Loadings 



Item 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 


1 


.01 


.42 


-.13 


.02 


•J2 


2 


.40 


.23 


-.02 


-.15 


• 14 


3 


-.08 


.3.1 


-.09 


.43 


•39 


4 


-.03 




.07 




.46 


5 


.35 


.04 


.20 


-.03 




6 


.43 


-.04 


-.01 


.29 


-.19 


7 

8 


-.03 

. 26 


.52 
♦ 34 


.09 

.07 


.05 

.10 


.00 

.04 


9 


.17 


.47 


.06 


.26 


.10 


10 


.45 


♦1ST 


.17 


.11 


.07 


11 


.08 


.15 


.08 


.48 


-.08 


12 


.46 


-.10 


.13 


.22 


.23 


13 


.29 


.17 


.06 


♦32 


-.04 


T 1 


27 


72- 


?7 




.73 


-i r 

up 


. 42 


.16 


*i *2 




• 


16 


. 42 


.16 


.18 


.22 


.11 


17 


.29 


.08 


.15 


.10 


.04 


18 


.17 


-.09 


.10 


-.21 


.55 


19 


.16 


. 44 


. .09 


.04 


.14 


20 


,08 


-.■Off* 


.04 


•12- 


.00 


21 


.55 


.08 


-.11 


-.03 


.02 


22 


.12 


.48 


.14 


-.05 


-.19 


23 


,.01 


.17 


.52 


-.17 


-.01 


24 


.07 


.33 


.41 


.06 


-.12 


f-N r* 

2p 


.47 


M 


-.00 


-.13 


-.07 


26 


.10 


.03 


.50 


.06 


. 22 


27 


.19 


. 46 


.08 


.02 


.07 


28 


.04 


-.14 


.52 


.12 


.10 


29 


.01 


.16 




.11 


-.02 


30 


. 20 


.09 




.02 


-.01 




Percent 


of variance 


(from Principal 


Components) 






12. 7% 


4.4% 


4.1% 


3.8% 


3.7: 




3j 
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Table 10 

Initial Licensing Rules Test (Form # 5) 
Rotated Varimax Factor Loadings 



Item . 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


1 


.37 


.39 


-.07 


; 08 


.12 


2 


.34 


.25 


-.06 


-.03 


.30 


3 


.IT 


.12 


.35 


.02 


-.08 


4 


.21 


.24 


.152; 


.37 


-.13 


5 


.23 


.32 


.01 


.■26 


-.12 


6 


.49 


.25 


~. 06 


.06 


.19 


7 


.39 


.05 


.23 


.22 


-.06 


8 


-To 


.29 


.10 


.04 


.14 


9 


.17 


.33 


.29 


-.01 


-.23 


10 


. 28 


.25 


.33 


-.05 


.10 


11 


.11 


. 04 




.12 


.10 


18 


.03 


.41 


.20 


-.04 


.16 




~.oa 


.60 


07 


- os 




14 


.61 


.14 


.10 


_ no • 


. 07 


15 

16 


.01. 

.27 


.12 

.39 


.48 

.03 


.14 

.12 


-.01 

-.08 


17 


.18 


-.03 


.44 


-.06 


.17 


18 


-.05 


. 44 


.16 


.23 


-.13 


19 


.19 


.32 


.06 ' 


-.08 


.4£ 


20 


.07 


.'02 


.11 


.75 


.11 


21 


.39 


.03 


.28 


.21 


-.20 


22 


-.09 


.02 


.56 


.06 


.06 


23 

24 


.11 

.61 


-.08 

-.00 


.02 

.13 


.18 

-.03 


.49 

.06 


25 




.05 


.03 


.09 


.07 


26 


.14 


.04 


. 25 


-.00 


.50 


27 


.56 


-.07 


. 06 


-.02 


.20 


28 


• 34 


.03 


. 26 


.04 


-.05 


29 


-.Ife 


-.02 


.12 


.75 


.14 


30 


.40 


.09 


.26 


-.00 


-.32 




Percent of 


variance 


(from Principal 


Components) 






14.5% 


5.3% 


4.1% 


3.7% 


3.6% 
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an adequate representation of the pattern and structure of these tests. In 
general, the factor patterns were similar across test forms in the variance 
accounted for, in the presence of one distinctly dominant factor and in the 
uniform uninterpret ability of the factor patterns. There was, unfortunately, 
no clear meaning derivable from the item content that could be assigned to 

* 

any of the item groupings, for any of the factors, on any of the test forms. 
This result is not entirely surprising if (as is likely) the test items were 
selected solely to provide maximum coverage of the material in the Driver's 
Handbook and with no a priori intention of selecting sets of items that 
cover particular subject-matter areas (e.g., use of signals, maintenance of 
distance, emergency procedures, etc.). 

Since there are no clearly definable, underlying, dimensions that appear 
to be tapped by these rules test forms, there is little guidance provided 
for identifying the most meaningful areas of driver knowledge that might be 
used in developing additional useful items, or in building more coherent sub- 
scales with more useful scoring procedures. However, a broad interpretation 
is possible, given test forms that are primarily unifactor in structure; 
with the factor a somewhat general one that explains the largest proportion 
of variance, by far. The dominant factor can best be conceived of as one 
reflecting "general knowledge" of Handbook content resulting, primarily, 
from reading skill (or verbal prof iciency) --i . e . , the one area of competence 
common to acquisition of such knowledge and to its demonstration in test 

This lack of factor interpretability can be demonstrated by using the 
items of the sample test forms shown in Appendix A (initial Licensing Rules 
Test, Form #l) and the factor loadings of Table 6. The resulting frustration 
that is encountered in attempting to assign meaning to the factors is 
identical over all five test forms . 
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performance „ What is suggested, in effects is that the test could be expected 
to rank driver applicants in a similar way if it were based on the abilit’ bo 
read and comprehend a cookbook*. (Some indirect evidence to support such an' 
interpretation is provided by significant and moderate levels of positive corre- 
lation found between the renewal licensing rules test scores and the applicant’s 
level of formal education, that are to be reported in a subsequent section.) 

The more critical question to be considered is, of course, whether or 

not the rules test is related to any aspects of driver performance. If 

the assumption is correct that the major skill component measured by these 

rules tests is heavily dependent on verbal proficiency then, on the basis 

of previous findings regarding the minimal relationship of intellectual 

skill to driver performance (Goldstein, 1961), there would be little reason 

to expect iisable levels cf correlation between rules test scores and post- 

* 

test driver performance. 

4* . Form Comparability : From visual inspection of the item character- 

istics, reliability estimates and factor structure it is apparent that a high 
degree of overall similarity exists among the five initial licensing rules 
test forms . Some differences that are relatively minor can be noted, for 
example, in contrasting forms $1 and 2 with the remaining three forms. 

Those two are most similar to one another in their distribution of item 
difficulties, their item-total test score correlations and their factor 
structure: while forms #3j> #4-, and #5 form a distinctly more comparable 
grouping. Again, however, such relatively minor differences in comparability 

* 

The planned report supplement dealing with an analysis of the relation- 
ship between the rules test and one year, posttest, accident and violation 
records will, of course, provide more direct evidence regarding this expectation 
than is possible with the present data. 

O 
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Table 11 

Number of Common Items between Test Forms 
Initial Licensing Rules Test 





1 


2 


Form jf 
3 


k 


5 


1 




5 


6 


6 


6 


2 






8 


6 


10 


3 








11 


5 


k 










7 


5 


i 
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would be expected if items from a pool were assigned bo test forms on a 
relatively random basis and the forms administered to applicants without 
systematic bias. One complicating aspect of any assessment of form compara- 
bility for this exam, is that much of it is built-in, or art i factual, and can 
be attributed to the number of common items that occur between forms . In 
Table 11, the amount of item overlap is shown for the five initial licensing 
rules test forms. Wo fewer than 5 (l6$) and as many as 11 (56$) of the 
items can be seen as common to any two test forms. 

Renewal Licensing Rales Test (Five Written Forms) 

In' Tables 12 through l6,the item characteristics and test reliability 
estimates are presented for the five forms of the renewal licensing examina- 
tion. The rotated factor loadings, and accountable variance for each factor 
appear in Tables 17 to 21 for the same five forms. Item selection proportions 
are, again, appended (Appendix C) as information for possible use in modify- 
ing the existing items . 

1. Item Characteristics : Item difficulty levels (means) shown in the 

first column of each table provide much of the basis for any expected value 
(or lack of value) likely to be found in these tests. On the basis of the 

difficulty indices alone, these measures are patently far less acceptable 

than the initial licensing rules tests. Mean values are seen to be greater 
than .80 for a large proportion of the items in each test form (i°e., from 

no fewer than 12, to as many as l4 of the 20 items). 

From 7 to 10 of the 20 items in any form are excessively easy for 
the renew al applicants with resulting mean values of .90 or greater. The 
distorting effects of such distributions of item difficulty indices is so 
pronounced in its restriction of item variances (second column of each table) 
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Table 12 



Written Renewal Rules Test 
Item Characteristics (Form A) 

(N = 16S-) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1 . 


Traffic signs 


.69 


.21 


.41 


2. 


License suspension rules 




.24 


.36 


3. 


Passing regulations 


.81 


.15 


.35 


4. 


Speed regulations 


.64 


.23 


.4o 


5. 


Speed regulations 


.91 


.08 


.42 


6. 


Safe driver attitude 


.90 


.09 


.48 


7. 


Speed regulations 


.49 


.25 


.24 


8. 


Intersection regulations 


.62 


.23 


.42 


9. 


Intersection regulations 


.95 


.04 


.43 


10. 


Use of lights 


.93 


.06 


.43 


li. 


Road markings 


.95 


.05 


.52 


12. 


Pedestrian right of way 


.85 


.12 


• 35 


13- 


Passing 


• 94 


.06 


.42 


Ik-. 


Braking distance 


• 53 


.25 


.39 


15. 


Use of lights 


.95 


.05 


.42 


16 . 


License re gulat i ons 


• 97 


.03 


.*7 


17. 


Road markings 


•76 


.19 


• 45 


18 . 


Speed regulations 


.79 


.17 


.33 


19. 


Intersection regulations 


• 91 


.08 


.42 


20. 


Passing regulations 


• 94 


.06 


.4o 



Coefficient alpha = .64 

Total Test Mean = 16.0 
Total Test SD =2.7 
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Table 13 

Written Renewal Rules Test 
Item Characteristics (Form B) 

(N = 163k) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1. 


Speed regulations 


.81 


.16 


• 37 


0 

t_ * 


Traffic signs 


.86 


.12 


.k8 


3* 


Turning regulations 


.85 


.13 


.37 


k. 


Interstate highway regulations 


.96 


.01+ 


.kk 


5. 


Traffic signs 


.8k 


.13 


.k2 


6. 


School bus law 


• 99 


.01 


.35 


7. 


License suspension rules 


.43 


.2k 


.ko 


8. 


Safe following distance 


.69 


.21 


.39 


9- 


Traffic signs 


.88 


.11 


.3k 


10. 


Traffic lights 


.90 


.09 


,k2 


11. 


Tt Saf ety-Responsibility Law” 


.76 


.18 


.50 


12. 


Pedestrian right of way 


.8k 


.13 


.38 


13. 


Intersection regulations 


• 91 


.08 


.kk 


lk. 


Speed regulations 


.50 


.25 


.51 


15. 


Use of lights 


• 93 


.06 


.38 


16. 


License suspension rules 


•79 


.17 


• 3k 


17. 


Safe driver attitude 


.91 


.08 


.k6 


18. 


Wight driving safety 


•71 


.21 


• 53 


19. 


Speed regulations 


■ 92 


.07 


.39 


20. 


Passing 


• 92 


.08 


.30 



Coefficient alpha = .69 

Total Test Mean = l6.k 
Total Test SD = 2.9 
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Table l4 

Written Renewal Rules Test 
Item Characteristics (Form C) 

(N = 1662) 



Item 



1 . 

2 . 

3. 

4. 

5 . 

6 . 

7. 

8 . 

9- 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 



Speed regulations 

Traffic signs 

Intersection regulations 

Interstate highway accidents 

Use of lights 

Intersection regulations 

Safe following distance 

Hand signals 

School bus laws 

Use of lights 

Passing regulations 

Use of lights 

Accident reporting 

Speed regulations 

Intersection regulations 

Licensing rules 

Traffic circle regulations 

Night driving danger 

Clover-leaf regulations 

Safe driver attitude 



Total Test Mean = l6.6 
Total Test SD =2.7 



Mean 

(p) 


Variance 


Item- 
Total r 


.80 


.16 


.41 


• 77 


.18 


.41 


.80 


.16 


.43 


.61 


.24 


.42 


.93 


.25 


• 27 


,90 


.09 


.43 


.83 


.14 


.48 


.94 


.05 


.31 


• 99 


.01 


.34 


.95 


.05 


.36 


.94 


.06 


.28 


• 54 


.25 


.44 


.87 


.11 


.41 


• 97 


.03 


.43 


.88 


.11 


.43 


.98 


.02 


.32 


.75 


.19 


.44 


.86 


.12 


.45 


.78 


.17 


.44 


.92 


.07 


.41 



Coefficient alpha = .66 
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Table 15 

Written Renewal Rules- Test 
Item Characteristics (Form D) 
(N = l6l4) 





Item 


Mean 

(p) 


Variance 


Item- 
Total r 


1 . 


Passing regulations 


• 79 


.17 


.40 


2. 


Traffic lights 


• 90 


.09 


• 37 


3. 


Intersection regulations 


• 91 


.09 


•37 


4. 


Point system penalties 


.62 


.24 


.36 


5. 


School bus laws 


.99 


.01 


.29 


6. 


Speed regulations 


.56 


.25 


.41 


7- 


Response to police emerg. vehs 


.83 


.14 


.26 


8. 


Accident frequency (statistics) 


• 75 


.19 


.35 


9. 


Changing lanes 


• 90 


.09 


.37 


10 . 


Passing regulations 


.86 


.12 


• 35 


11. 


Accident reporting 


.88 


.11 


.36 


12. 


Licensing rules 


• 97 


.03 


.4o 


13- 


Traffic signs 


.87 


.11 


• 49 


14. 


Intersection right of way 


.96 


.04 


.46 


15c 


Intersection right of way 


.81 


.16 


.44 


16 . 


Use of lights 


.56 


.25 


.39 


17. 


Road markings 


.96 


.04 


.34 


18 . 


Safe driver attitude 


.91 


.08 


.49 


19. 


Speed regulations 


.92 


.08 


• 36 


20. 


Intersection regulations 


• 59 


.24 


.44 



Coefficient alpha = .65 

Total Test Mean = 16.5 
Total Test SD = 2.6 
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Table 16 

Written Renewal Rules Test 
Item Characteristics (Form E) 

(n = 1623) 





Item 


Mean 

(eI 


Variance 


Item- 
Total r 


1 . 


Speed regulations 


.86 


.12 


• 34 


2. 


Hand signals 


.95 


.05 


• 32 


3. 


Accident reporting 


•73 


.20 


.49 


4. 


Safe following distance 


.82 


.15 


• 52 


5. 


Intersection regulations 


• 9^ 


.06 


.28 


6. 


School bus laws 


• 99 


.01 


.26 


7- 


Passing regulations 


•77 


.18 


.42 


8. 


Braking distance 


.64 


.23 


.44 


9- 


Turning regulations 


.88 


.11 


.33 


10. 


Licensing rules 


.98 


.02 


• 27 


11. 


Use of lights 


• 95 


.05 


,4i 


12. 


Braking distance 


.49 


.25 


.38 


13. 


Interstate highway regulations 


• 95 


.04 


A3 


14. 


Turning regulations 


• 90 


.09 


.44 


15 . 


Use of lights 


.93 


.06 


.39 


16. 


Safe driver attitude 


.90 


.09 


• 50 


17. 


Speed regulations 


.43 


.25 


.33 


18. 


Point system schedule 


.61 


.24 


• 37 


19. 


Intersection regulations 


.64 


.23 


.42 


20. 


Response to police emerg. sigs 


.84 


.13 


.20 



Coefficient alpha = .62 

Total Test Mean = 16.2 
Total Test SD = 2.6 
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that -^ -tually all subsequent data analyses applied are likely to show 
adver:..- test characteristics. The source of the weaknesses in item 
characteristics are seen most immediately in the selection proportions 
of item alternatives (Appendix B) b There is little question that large 
numbers of distractors serve no useful purpose when incorporated in these 
multiple-choice items. A variety of test parameters are likely to be dis- 
torted by such unfavorable item values and severely diminish the opportunity 
to demonstrate any "true” effectiveness that (with little more effort in 
item development) might have been possible for renewal test items of this type. 

Any explanation, or justification, that attributes these item difficulty 
levels to the "superior" level of knowledge possessed by North Carolina renewal 
license applicants, might be difficult to support. In order to do so ; one 
would have to argue for the "inherent” value of the knowledge of these 
particular rules and regulations and ignore completely any evidence dealing 
with whether or not a given level of such knowledge is sufficient to lead 
to acceptable driver performance in any way. 

2. Form Reliabilities : The values are found to range over the low to 

high .60 T s for all five test forms. When contrasted with the initial licensing 
tests , this reduction in degree of internal consistency is most likely 
attributable to the difference in test length (i.e., 20 items as compared to 
30). s ince somewhat higher levels of reliability wou d be desirable, a 
longer renewal examination (probably 30 items or more in length) would be 
of value. However, given the greater detriment to test quality imposed by 
the item characteristics, reliability, as such, is of minor concern here for 
judging the adequacy of these measures. 
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3. Test Factors : The pattern of loadings for five rotated factors 

extracted from each test (Tables l 6 through 21 ) are not only uninterpretable 
for any of the test forms, but they lack even the structural uniformity of 

a general or dominant factor as clear as the one that appeared in the initial 
licensing test forms. The dimensional features of these forms--based primarily 
on the number and magnitude of interpretable loadings for the five factors 
(i.e., in excess of . 30 ) — are also seen as considerably different from form-to- 
form. Lack of interpretive qualities (i.e., meaningful item subgroups), or compar- 
ability of factor structure , can be attributed largely to the item inadequacies 
already mentioned; particularly the minimal amount of item variance available. 

4. Form Comparability - - is difficult to define meaningfully for these 
five renewal licensing test forms, except insofar as tney can be considered 
"comparably-less-than-adequate " to distinguish between levels of driver 
knowledge of Handbook content. 

Although the five forms are highly similar in their item characteristics and 
overall levels of reliability, much of this is built in (as in the case of 
the initial licensing rules tests). The extent of this item duplication, 
or overlap, is shown in Table 22 which indicates that anywhere from three to 
nine of the 20 test items are common from form to form. As will be seen below 
in discussion of the relationships between test forms, the driver background 
characteristics and the driver record (i.e., prior accidents and violations 
incurred), the pattern of correlations is also found to be fairly similar 
across test forms. 

5 . Renewal Rules Test Validity --obtainable from the correlations of 
the rules test scores with the driver's accident and violation occurrences, 
represents the initial opportunity to check the value of this written test 
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Table 17 

Renewal Licensing Rules Test (Form A) 
Rotated Varimax Factor Loadings 



Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 



,08 


. 5A. 


.18 


-.07 


-.18 


.15 


•aa 




.06 


-.01 


.10 


-.19 


.21 


.15 


. 04 


.21 


.06 


.49 


~. 24 


.11 


» 38 


. 14 


. 28 


.15 


-.3£ 


.45 


.33 


.12 


-. 06 


-.00 


.oS* 


.04 


.13 


.01 


.88 


.12 


. 23 


. 4-t 


-.03 


— . 04 


.62 _ 


.06 


.07 


-.06 


.06 


- 58 


. jl 7 


. 6-4 


-.19. 


-,I7 


'.70 


.10 


.12 


-. 06 


— . 04 


.25 


.44 


-.10 


-.08 


,12 


.46 


.28 


-.03 


.09 


.10 


..'03 


-.59 


.09 


,20 


. 11 


.57 


.12 


-.03 ’ 


. 1 4 


,03 


."57 


.02 


.03 


po 

. 


.04 


.20 


. 44 


.09 


. 23 


-.10 


.10 


.XT' 


-.03 


.80 


-.03 


.49 


— . 04 


.23 


.07 


-.18 


.52 


-.08 


.10 


.37 


. 01 


Percent 


of variance 


(from Principal 


Components) 




19 . 0% 


6.9% 


5.8% 


5.2% 


5. 



Table l8 



Renewal Licensing Rules Test (Form B) 
Rotated Varimax Factor Loadings 



Item 


•Factor 1 


Factor 2 


Factor 3 F 


actor 4 


Factor 5 


1 


.09 


-.03 


.30 


.02 


.54 


2 


.23 


.61 


.01 


.07 


.18 


3 


.11 


.08 


-.03 


.15 


.62 


4 


.56 


. 29 


-.01 


.06 


.09 


5 


.34 


.25 


-.08 


.31 


.03 


6 


.62 


-.08 


.09. 


.04 


. 21 


7 


-.12 


. 22 


.17 


. 64 


-.04 


8 


-.01 


.49 


.10 


-.01 


.11 


9 


.07 


•32 


-.03 


-.08 


.52 


— 










'S 


11 


. 20 


.37 


.24 ' 


. 29 


-.03 


12' 


.10 


.07 


. 60 


-.04 


.08 


3-3 


.51 


.20 


.18 


.03 


-.01 


14 


-.06 


. 44 


.27 


•25 


.13 


15 




— . 14 


.15 . 


.13 


.15 


16 


. 23 


-.20 


-.07 




.17 


•17 


.37 


.25 




.20 


-.13 


18 


-.04 


.28 


.57 


.09 


•34 


19 


. 23 


-.04 


.33 


.18 


- 40 1 


20 


.36 


.04 


/2? 


-.16 


-.01 




Percent 


of variance 


(from Principal 


Components ) 






17. 7% 


9.7% 


5.5% 


5.3% 


5.1% 
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Table 19 

Renewal Licensing Rules Test (Form C) 
Rotated Varimax Factor Loadings 



Item 


Factor 1 


Factor 2 


Factor 3 


Factor 4 


Factor 5 


i 


. 15 


.29 


.21 


.21 


-.44 


2 


-.03 


.31 


.45 


-.00 


- o 00 


3 


.02 


.62 


.02 


.15 


-.09 


4 


.09 


. 25 


° 39 


.04 


-.29 


5 


-.14 


.13 


-.21 


.57 


.23 


6 


.40 


.25 


-.00 


.33 


- .12 


7 


.01 


.56 


.30 


.09 


-.01 


8 


,61 


-.03 


.04 


.05 


-.04 


- 


iL 


-.o:> 




. O'li 


— „ 6o 


10 


. 28 


.35 


-.00 


.12 


.38 


11 


.09 


.10 


.23 


.04 


.63 


12 


-.01 


-.02 


.70 


.09 


.02 


13 


.28 


.09 


.43 


-.01 


. 33_ 


14 


.64 


.16 


-.04 


.14 


. 16 


15 


•34, 


.38 


.21 


-. 11 


-.02 


16 


.49 


.20 


.05 


-.09 


.18 


17 


. 15 


-.01 


.21 


.62 


-.05 


IS 


.22 


.04 


.47 


. 21 


. 14 


19 


.07 


.03 


.18 


- .65 


-.08 


20 


.14 




.00 


. of 


.23 




Percen t 


of variance 


(from P v in c i p a 1 Compoiien to ) 




17.0% 


7.0% 


6.0% 


5.5% 


5.0% 
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Table 20 

Renewal Licensing Rules Test (Form D) 
Rotated Varimax Factor Loadings 



Item 


Factor 1 


Factor 2 


Factor 


3 Factor 1* 


Factor 5 


1 

2 


* 13 
-11* 


..37 

.58 


olt2 

.03 


-.20 

-.02 


-.03 

.10 




G 0^ 


-1*3 


o32 


.00 


-.03 


k 


.01 


-.00 


.13“ 


.12 


.79 


£ 


.61 


-.02 


0 1 6 


-.10 


-.02 


6 


.09 


.30 


.02 


-.12 


.57 


7 

c 


.30 


-.30 


-.Oli 


.1*5 


.26 


8 


.33 


M 


-.21 


-30. 


0/1 


9 


.30 


.12 


.16 


.29 - 


0 n 

— • j 


10 


.08 


.10 


.25 


.no 


.01 


1 1 


.18 


“*13 


.70 


.11* 


.06 


12 


.68 


.11 


.02 


.08 


.01 


13 


.1*3 


.27 


.31 


-.05 


.10 


11* 


.62 


18 


.on 


.15 


.10 


15 


.08 


.50 


.07 


.25 


-.00 


16 


-.05 


oi9 


.07 


.65 


-.05 


17 


.32 


»37_ 


-.02 


-.00 


.01 


18 


0 33 


a @:_. 


.12 


.10 


.10 


19 


-.02 


. i 6 


.52 


' 10 
• 1 / 


.12 


20 


-.03 


-55 


-.755" 


.26 


.11 




Percent 


of variance 


(from Pr 


incipal Components) 






16.1% 


6.3% 


5.6% 


c. '> *7 

J • £>/« 


5.1% 
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Table 21 

Renewal Licensing Rules Test (Form E) 
Rotated Varimax Factor Loadings 



Item 


Factor 1 


Factor 2 


Factor 3 


Factor 2i 


Factor- 


1 


.00 


.19 


.05 


-.62 


-o05 


2 


.21 


.2*3 


o07 


.02 


.02 


3 


•12 


.752 


•3k 


.35 


-.08 


h 


.2*2* 


.12 


.11 


.39 


.03 


5 


-.(53 


• 6 II 


-.09 


•33 


-.09 


6 


.06 


.66 


.01 


-.00 


.18 
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.20 
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.2*9 


-.08 


.03 


.17 


-.15 
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-.03 


.16 


.12 


- . 1 6 


.U7 




Percent 


of variance 


(from Principal Components) 






16.0% 


6.6% 


6.1% 


5.5% 


5.3% 
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Table 22 

Number of Common Items between Test Forms 
Renewal Licensing Rules Test 

Form 



B C JD 



A 




4 


8 


7 


8 


B 






6 


4 


3 ! 


C 








... - | 

8 


6 


D 








i 

i 9 I 

S 1 


E 








1 s 

i 

1 i 

I 1 
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against an external performance standard. These product -moment correlations 
are shown in Table 25 for each of the five test forms by male and female 
subgroups . 

As might have been expected from the results discussed above, concerning 
poor internal measurement characteristics for these renewal test forms 
(especially the highly restricted item variance), only minimal levels of 
relationship are found. Two of the 20 correlations reach a level signifi- 
cantly greater than zero and both of these occur with test form C. On that 
test form there is a very slight (and barely significant) tendency for those 
female applicants scoring high on the test to have incurred fewer violation 
convictions (r ~ -„10, p < .05 )• The one significant relationship for males 
is found for the criterion of accident occurrences (r = -.16, p < .01). It 
is not immediately clear from inspection of item characteristics or content 
why test form C should show such tendencies toward significant relationships 
with the criteria. But, in either event, these relationships are at levels 
that preclude any practical utility. The r of . 16 , for example, would 
provide less than a 5 °l° improvement in predictive accuracy over a test having 
no relationship to the criteria. One weak, but hopeful, sign for potential 
posttest predictive validity (if, these renewal exams were to be constructed 
with more defensible item characteristics), is that the largest of the 
correlations, especially those for the accident criterion, are all in the 
expected direction (i.e., all are negative in sign). 

Attempts to enhance the set of relationships by using the accidents and 
violations in optimally weighted combination (Table 24) do not produce any 
notable change for the multiple correlations obtained. The highest R of .16 
(p < .001) for males on test form C is no better than the relationship achieved 
for the zero-order coefficients. 
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Table 23 

Correlation Coefficients for Renewal License Rules Test 
with Driver Accidents and Violations 
(By Sex and Test Form) 



Test 


Form 




Violations 


Accidents 


N 


A 


Male 




.01 


- .06 


522 




Female 




.02 


.04 


368 


B 


Male 




.02 


- .08 


522 




Female 




-.08 


-.02 


396 


C 


Male 




-.08 


-.16 


513 




Female 




* 

-.10 


-.09 


404 


D 


Male 




-.01 


-.05 


535 




Female 




.01 


-.08 


352 


E 


Male 




-.05 


-.03 


490 




Female 




-.08 


-.05 


372 




* 

Significant 


at 


.05 confidence 


level . 






** 

Significant 


at 


.01 confidence 


level. 
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Table 24 

Multiple Correlations for Renewal Rules Test Scores 
Regressed on Accidents and Violations 
(By S ex and Test Form) 



Test 


Form 


Accidents and Violations 


A 


Male 


.06 




Female 


.o4 


B 


Male 


.10 




Female 


.08 


C 


Male 


.* 

.16 


• 


Female 


• ii 


D 


Male 


.05 




Female 


.10 


E 


Male 


.05 




Female 


• r 9 




* 

R significant 


at p < .001 confidence level. 
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It is only when the background data are combined with the rules test 
scores, as predictors of a combined accident-violation criterion score, 
that some useful predictive possibilities emerge. Table 25 lists the 
canonical correlations obtainable from those best weighted combinations of 
the set of six predictor variables (five background variables plus the 
rules test) and the accident-violation criterion variables. These rela- 
tionships are shown in terms of maximum and minimum coefficients obtainable. 
The significant maximum canonicals (R max.) are of primu interest for 
discussion here. 

All rules test forms, when combined with the five background variables 
for males, result in highly significant maximum R's (p < *001 ) that reach 
moderate levels in the .30 f s. The variance accounted for (or the extent 
to which prediction would be enhanced by R's of such magnitude) ranges 
from approximately 10# to 15#. This represents an improvement in predictive 
accuracy of b to 5 fold as a result of combining background data with the 
rules test. By contrast, the addition of background variables for the 
female samples produces only one significant relationship with the accident - 
violation criterion (R = .25; p ^ *01 for test form C). The fairly large 
differences in the size of these R's for males and females make it apparent 
that the background data are far more effective predictors for a male sample 
of drivers than for females — a finding that has, generally; been supported 
in several studies undertaken for the State of California (Coppin et al. 7 

The set of minimum canonical R's (R min.) represent the lowest likely- 
set of relationships achievable from the fit of these predictors to the 
criteria. Maximum (or first) canonicals (R max.) tend to be somewhat 
inflated estimates that are not likely to be reproduced in cross-validation 
at quite such high levels. 
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Table 25 

Canonical Correlations for Renewal Rules Test Applicants 
Composite Accidents and Violations with Renewal Rules 
Test Scores and Five Background Variables 
(By Sex and Test Form) 



Tes t 


Form 




R 

max 


R . 
mm 


A 


Male 




.36 


.10 




Female 




• 15 


.11 


B 


Male 




** 

.39 


* 

.17 




Female 




• 19 


.13 


C 


Male 




** 

.33 


* 

.19 




Female 




* 

.25 


.12 


■ D 


Male 




, ** 
.34 


.14 




Female 




.22 


.14 


E 


Male 




** 

• 32 


.09 




Female 




.18 


.06 




* 

Significant 


at 


p < .01 


c on f i de nc e 1 ev e 1 * 




Significant 


at 


p < .001 


confidence level* 
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1967 ). Further support for that conclusion is seen in the values of the 

minimum obtainable canonical R's, wherein the only two significant corre- 

. -x- 

lations obtained are for males on test forms B (R = -17) a-nd C (R = -19) • 

In esp p "" , .e, then, by combining rules test scores with background data, 
there - be reasonable confidence that, for males, correlations signifi- 
cantly better than zero (i.e., no relationship) can be achieved in predicting 
a composite accident-violation criterion. Variance explained by such 
predictors would be expected to range from about 5.5$ to a maximum of 15$* 
Although the predictive advantage at such levels can oe used cautiously 
for certain decisions regarding drivers. (e.g. , classification, guidance and 
educational needs), these variables remain of limited practical /alue in a 
selection or screening system whereby some form of cutoff score must be 
arrived at in deciding whether or not a license should be granted. Levels 
of predictive efficiency, for even the largest of the correlations found when 
the renewal exams are combined with background data, would result in denial 
of the renewal license to too many "good" drivers in order to eliminate a 
reasonable proportion of those who are likely to incur accidents and/or 

f 

violations . 

Despite the limited practical utility in evidence here, an equal 
advantage in deriving information of this sort lies in its value for improv- 
ing the understanding of important characteristics of the driver population 
and for providing clues to more fruitful lines of research and development 




*The differences in predictive value by sex can be attributed, in part, 
to a statistical artifact that arises from (a) the extremely skewed (Poisson) 
distribution of accident and violation frequencies and (b) the significantly 
differing frequency of accident and violation occurrences for the two groups. 
Males drive more than females and incur more accidents and violations 
(especially accidents). This higher frequency of occurrence for males tends 
to increase accident-violation criterion score variability and, concommitantly, 
to enhance the stability (or reliability) of that _ criterion. The net effect 
is a tendency to enhance the potential predictability of these criteria for 

males . 

55 



-53- 



•fchat bear on program operation. It was clear, for example, that supple- 
mentary (background) data combined with the rules test scores were needed 
in order to raise the correlations to levels with any reasonable predi' ti/u 
value . A better understanding of those multivariate relationships and the 
influences of driver characteristics that help produce them can he arrived 
at by examining the intercorrelations between the rules test score, the five 
background variables *and the accident and violation criteria. 

The two 9 x 9 intercorrelation matrices are presented in Tables 26 and 
27 for males and females respectively. For convenience of presentation 
the matrices selected incorporate the results obtained only with rules test 
form A, since these were considered reasonably representative of the matrices 
obtained for each of the other four test forms. Some of the reasons for the 
more extensive impact of background variables , as performance predictors for 
male drivers, become apparent when the two matrices are compared. 

It can be seen from the number and pattern of significant relationships 

* 

that there is greater potential influence of individual background variables 
for males on the accident and violation criteria. Thus, driving experience 
for males ("years of driving") is significantly related to accidents and 
violations (or both used additively) and to the rules test scores. The 
variable of age which, as expected, is very highly correlated with years of 
driving experience (r = - 9 ^) > results in an almost identical pattern of 
significant correlations. In effect, those males who are older and have 
been driving for a longer perio " - ' time are clearly 'better drivers (in 
terms of recorded accidents and violations) but poorer test takers. Such 
relationships do not appear cons ' stently for the female driver sample, and 
the only significant correlate )n found (r = -.10) indicates a slight tendo-^oy 

for younger females to be convicted of more violations. 
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Intercorrelation of Background Variables., Accident and Violation 
Frequency and Scores on Renewal Rules Test Form A 
(Males , N = 522) 
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Relationships found between formal education for males (years of school) 
and several driver characteristics are also helpful in understanding the 
rules test results and driver performance. Those males with more formal 
education tend to be the ones who score higher on the rules test (in support 
of the previously stated assumption that the measure is, to a significant 
extent, one of verbal-intellectual skill). Males with more formal education 
also tend to be younger, to drive newer cars and to have been driving for 
fewer years , Probably because of the relationships of age and driving 
experience to the criteria (as previously shown) those with more education 
also tend to incur more violations (r = .11* p < .05) and more accidents plus 
violations (r - *11, p < *05)* For females, the pattern is somewhat similar, 
with the more educated females scoring higher on the rules test, tending to 
be younger and to drdve newer cars . Unlike the results for males, however, 
formal education is unrelated to violation convictions or accidents* 

Ihcposure, as defined by "miles driven per week, ?I relates primarily to 
the number of violations incurred by male drivers (r = .12, p < .01) but is 
unrelated to either accidents or violations for females. A reasonable 
suspicion, stemming from the pattern of weak relationships for miles driven 
over any period of time, is that it represents an incomplete index of 
exposure to potential accident or violation producing situations and should 
be supplemented by variables that take into account the traffic density likely 
to be encountered by each driver (e.g., population of the area where most of 
the miles are driven; number of vehicles registered in that area). 

The age of the automobile as a background variable is found to bear some 
minor relation t accident- violation performance for both males and females, 
although at low levels of significance (p < .05). These results offer some 
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support to the logical expectation that more accidents and violations occur 
for persons driving older cars. 

In interpreting any of the relationships presented above, it is necessary 
to recall that validation of these renewal rules test forms are dependent on 
a retrospective analysis and the addition of relatively few background vari 4 - 
ables . It should also ba kept in mind, that the rules test measure itself 
was expected to hold little promise in demonstrating meaningful correlations 
with complex driver performance criteria because of the relatively poor 
quality of its measurement characteristics. In addition, total dependence 
on accident and violation criteria, with their highly skewed distributions, 
generate statistical properties which make for difficulties in analyses and 
interpretation whatever the study design. Nevertheless, within these con- 
straints, the relationships did form fairly logical and interpretable patterns 
that are indicative of at least some degree of consistency and stability for 
the limited range of predictor and criterion variables available* The 
implication is that incorporation of measures of better quality and wider 
variety could have enhanced the level of relationships and sharpened the 
patterns found here. 



CONCLUSIONS AND RECOMMENDATIONS 



These initial analyses of written rules tests , presently used for 
licensing purposes in the State of North Carolina, have indicated that: 

(l) The adequacy of the rules tests, as measurement tools for purposes 
of granting or renewing a driver's license, leaves room for considerable 
improvement. On the basis of item analyses, the various forms of the rules 
test tended to contain too large a proportion of items that are excessively 
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easy for the license applicants^ with serious effects on item and test 
variance. These shortcomings were found to be particularly true of the 
license renewal examination. However ^ levels of internal consistency 
(reliability) for the measures were found to be adequate and reasonably 
similar from test form to test form* As a concomitant of that consistency^ 
most of the items appeared to be measuring similar aspects of applicant 
knowledge which was reflected in moderately high levels of item-total test 
score correlations. Just what particular knowledge or skill areas are being 
measured was , unfortunately^ not clear from the item groupings obtained as 
a result of factor analyses. Factor patterns for the rules tests were almost 
entirely uninterpretable and there appeared to be no coherent,, identifiable^ 
areas of driver knowledge or skill being measured* The one possible exception 
was a dominant factor ^found for each of the initial licensing rules test forms , 
that was tentatively identified as ^general knowledge” of the driver handbook 
and was believed to stem primarily from applicant verbal-intellectual skills * 
For the written renewal exam* the factor patterns were neither similar from 
form to form nor interpret able in any way* 

(2) The retrospective ^cr post hoc 3 validation of the renewal rules test 
forms was limited to criteria based solely on driver records of accidents 
and violations for a four -year period prior to the administration of the 
renewal examination. Logical problems were stressed in generalising from 
after-the-fact validity analyses to predictive assumptions about the measures , 
Accepting those limitations j however * the findings indicated that only one 
renewal test form (Form c) showed any degree of potential value (although 
slight) by virtue of its significant correlation with violation convictions 
for females and with accident occurrences for males . The levels of 
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relatlonstp p are such that they provide little value for any practical pre- 
dictive application in driver appraisal* especially for screening purposes* 

(3) Only with the addition to the rules test scores of driver background 
data (i,e.* age* formal education* years of driving* age of car* and miles 
driven per week), were any reasonable levels of relationship with the 
accident-violation criteria found* Correlations based on the best weighted 
combination of accidents and violations as criteria and the best weighted 
combination of the rules test score plus background variables us predictors* 
produced maximum canonical R f s in the .30 f s that were uniformly significant 
across all test forms for the male driver applicants. Even at those levels 
of validity* the predictive value of the rules tests would be considered 
highly limited for the screening of drivers in terms of who should* or should 
not* be granted a license. Some bases for the multivariate results and the 
striking differences by sex in the potential predictive value of tests and 
background data become clearer from an examination of the univariate corre- 
lations. Thus* individual background variables in general- -especially a.ge 
and years of drivings -were found more consistently and more, highly correlated 
with accidents or violations (and the two scores combined) for males than for 
females . 

Any recommendations for test improvement* based on results achieved with 
the present data* must be considered preliminary* Their confirmation 
(or modification) would depend on initial licensing rules test validation in 
a predictive* or pospective* analysis (i*e** a posttest follow up) planned 
as a supplement to this report. Given that caution* the limited recommenda- 
tions applicable to the written rules tests are: 
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(1) There is a need to construct test items with more defensible measure- 
ment characteristics than were found in the present rules tests. A strong 
implication for any such recommendation is that a clear definition of the 
purposes of the written driver licensing tests can be arrived at* If the 
sole basis for test use is no be a definition of the extent to which an 
applicant can recall the content of the Driver’s Handbook; then proper 
content validation procedures for test construction w 5 ould be sufficient 
(Cronbach; 1971). If awareness of "good driving procedures" and "good 
Judgment" in vehicle handling are to be tested; then it is highly question- 
able that the present content of either the tests or the handbooks cover an 
adequate range of material appropriate to that purpose* If; on the other 
hand; the intention (however implicit) is to measure the extent to which 

the applicant will be a saf©; law abiding driver; then the present rules 
tests have not--as yet --demonstrated any degree of merit sufficient to 
warrant their use for that purpose. (Any hope of incorporating all three 
purposes in a single type of test may be open to serious question on practical 
grounds; since it would probably require a major overhaul of present test 
and handbook content- -as well as an extensive research program to determine 
what that content should be— before one could even begin to develop a 
suitably valid test.) 

(2) In any future approach to developing rules; or knowledge -Judg- 
ment measures there is a need to define specific and coherent content 
areas of dr iver knowledge that are intended for evaluation* The purpose 
is to build into the measure (s) meaningful item subgroups ^ or potentially 
useful subscales ; that would identify those forms of driver knowledge and 
Judgmental skills which are felt to be (or known to be) of importance to 
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driving p e r f onnane e and worthy of further developmental const derat ion * This* 
of course* implies 'being able to build measures based on more than reading 
ability* or verbal skills * as the single major component* 

(3) Since a significant portion of the improvements in rules test 
validity appeared to stem from the addition of background variables, it is 
suggested that a wider range of driver characteristics and driving experience 
(e-g- i> accident culpability* accident -violation severity* density of traffic 
customarily encountered* etc.) be incorporated for investigation in any 
further predictive studies of licensing tests* 

(t) There is a need to develop an assortment of more readily available* 
standardized measures of driver performance that can serve as defensible 
criteria* as opposed to depending entirely on recorded accident and violation 
data* Major statistical and logical difficulties in uses of accident and 
violation frequency have been pointed out in the present study and in 
previous research efforts (Coppin* McBride* & Peck* 1965) • When these diffi- 
culties are coupled with the time and cost often required to obtain such data* 
with sufficient accuracy over a long enough period of driver experience* it 
becomes apparent that a range of more immediately available (short-term) 
criterion measures is needed* Performance criteria, that are representative 
of driving capability (based on real-world or simulated environments ) and 
possessing more conventional measurement properties* would serve to provide 
the greater flexibility essential to test validation* 
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„es Tests 



Series “.A" 
(R*v. 8/63) 



North Carolina Department of Motor Vehicles 
DRIVER LICENSE DIVISION 



DRIVER EXAMINATION 



Read ALL possible answers. Some questions may have more than one answer that is partly right, but there is one 
best answer, Make a check beside the ONE BEST answer. The question will be marked wrong if there is more than 
one answer given,- Answer all questions, 



L Check the State maximum speed limit for passepger cars in a residential district, 
a r 25 mph b, 30 mph 35 mph cL 50 mph 

2. The lert arm held straight ou^rthe first finger pointings is the hand signal for 
a, a right turn left turn c, slowing down d. stopping 



3, The biggest difference between safe drivers and unsafe drivers is in 



age 

attitude toward highway safety 



c. physical condition 

d. mechanical ability 



4, V/hen following another car* you should 

a little more than one ear*length behind for every ten miles per nour of speed 
b* allow just enough distance so that an overtaking car can pull in ahead of you 
e, stay at least four car-lengths behind at all speeds 
d, keep 500 feet between you and the car ahead 



5, If two vehicles approach an intersection at th* 
vehicle on the right 
b. The vehicle traveling faster 



same time and from right angles* which one has the right of way? 

c. The v hide on the wider street 

d, The vehicle on the left 



6* When you are following a school bus that has stopped to take on or let off children, you should 

a, blow your horn, stop* make sure you can pass safely* and then go on 

b, slow your speed to 10 mph 

c, move to the left side of the highway and pass safely 
^di"come to a complete stop 

7- To pass on a three-lane highv/ay, you should use the 
a. center lane 

51 center lane if passing is permitted in the direction you are traveling 
c. extreme left-hand lane 
d* rightTiand lane 



8. Compared to the braking distance at 20 mph, the braking distance at 40 mph is 

a* six times as far four times as far e, twice as far d, no farther 



9- The law requires a turn signal to be given at least 

feet before the turn c, 

b„ 75 feet before the turn d. 



50 feet before the turn 
25 feet before the turn 



10* A person’s operator's license expires 
a, only when cancelled 

b* op his birthday one year after it is issued 
^efon his birthday four years after it is issued 
^ Three years after it is issued 



ERiC 



e& 



Series fl 5 A 4? continued' 



11, The^phrpose of the Safety^Responsibility Law is 

to make sure that the cos:s of accidents are borne by those responsible for them 



b* to lower insurance rates 
e. to reduce , seeding 
d. to raise insurance rates 



12. At intersections where there are no traffic lights, pedestrians have the right of way 

a. only if they are in marked crosswalks 

b. over vehicles which are turning, but not over those going straight ahead 

c . oyer vehicles going straight ahead, bur not over those wdiich are turning 
over all icles if the pedestrians are in marked or unmarked crosswalks 

13. What must you do as you approach a corner at which you intend to turn left? 
a- Pull over to the curb and stop until the street is clear 

b e Signal and get into the lane nearest the right curb 
^^elcugnal and get into the lane nearest the center line 

d. Speed up slightly 

14. Wh entente ring the traffic lane on an interstate highway , you should 
^a^enter the traffic lane at the same speed as the cars in that lane 

b. enter the traffic lane at a slightly faster speed then the traffic in that lane 

e. stop and make sure the traffic lane is clear 

d. enter the traffic lane at a slightly slower speed than the traffic in that lane 

15* While driving at night, if you meet a car with blinding headlights, the best thing to do is 
continue at the same speed, but do not look directly at the oncoming lights 
b. dip> your lights and speed up to get by quickly 
-^*slow down, dim your lights, and do not look directly at the oncoming lights 

d. drive with two wheels on the right shoulder of the road to give yourself more room while meeting the car 

16. A drivers license must be suspended for thirty days when He Is convicted of 

a, driving without a license c. failing to dim headlights 

^bf^driving an automobile more than 75 mph d. failing to yield right of way 

17. A person who drives in a manner that is likely to injure persons or property is guilty of 

a* drunken driving c. no^riolation 

b. driving without a license ^dt^reckless driving 

IB. A h§isic rule for safe night driving is 
never outrun your headlights 
b, drive at 35 mph 

c m drive in the middle of the road except when meeting other cars 
d. drive at the posted speed limit 

19. Very slow driving is especially dangerous 

a. in residential districts 

b. in business districts 

c. onjightly traveled country roads 

on highways just after passing the top of a hill, or just after rounding a curve 

20, Which is generally considered the most dangerous place to pass? 
a. Just beyond a hilltop 

At the bottom of a hill 

Going downhill 150 feet beyond the hilltop 
On any part of a hill where you cannot see 500 feet ahead 
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APPENDIX B 

Selection Proportions for Item Alternatives 
Initial Licensing lest Forms 1 to 5 
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Table B-l 

Initial Licensing Rules Test: Selection Proportions 

for Item Alternatives (Form $l) 



Item 


Alternative 1 


Alternative 2 


Alternative 3 


Alternative 4 


i 


.14 


.06 


.31 


,48* 


2 


.08 


.52* 


,36 


.02 


3 


.07 


.02 


.65* 


.26 


4 


.29 


.24 


.10 


,35* 


5 


.03 


.51 


.41* 


.04 


6 


.15 


.08 


,68* 


.07 


7 


,23 


. 70* 


.01 


.04 


8 


.91* 


.03 


.03 


.02 


9 


,66* 


.04 


.28 


,01 


10 


.03 


.14 


.49* 


.32 


11 


,04 


.81* 


.12 


,02 


12 


,25 


,53* 


.05 


.15 


13 


.76* 


.11 


,05 


.08 


14 


.08 


.02 


.85* 


.04 


15 


,30 


.53* 


,10 


,05 


16 


,63* 


.08 


,23 


.03 


1 7 


, 1 5 


. 5 3* 


. 23 


.06 


T O 

:,e 


.14 


. 81* 


.02 


.01 


19 


.04 


.08 


.17 


.70* 


20 


.05 


.11 


.70* 


,12 


21 


.86* 


.13 






22 


,60* 


.39 






23 


,82* 


,16 






24 


. 75* 


,23 






25 


,37 


. 62' ! 






26 


.30 


.69* 






27 


,13 


.86* 






28 


.63* 


.35 






29 


,38 


.61* 






30 


,91* 


.08 
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Table B-2 

Initial Licensing Rules Test: Selection Proportions 

for Item Alternatives ( Form #2) 



Item 


Alternative 1 


Alternative 2 


Alternative 3 


Alternative 4 


1 


.04 


.37 


.22 


. 37* 


2 


.07 


,79* 


.07 


,05 


3 


.58* 


.13 


.21 


.07 


4 


.07 


.03 


.02 


.88* 


5 


,02 


,66* 


.12 


.19 


6 


.12 


.02 


,29 


.56* 


7 


.07 


.02 


.83* 


.06 


8 


.05 


.05 


.84* 


.05 


9 


.08 


.08 


,21 


.61* 


10 


.14 


.57* 


.21 


.06 


11 


.21 


.58* 


,14 


.05 


12 


.12 


.01 


.85* 


.02 


13 


.21 


.58* 


.18 


,02 


14 


.05 


.14 


.68* 


.12 


15 


.87* 


.04 


.04 


.03 


16 


.07 


.13 


.56* 


.23 


17 


.16 


.63* 


.06 


.13 


18 


.94* 


,02 


.01 


,01 


19 


,47 


.34* 


.08 


,08 


20 


.10 


,15 


.50* 


.22 


21 


.15 


.84* 






22 


.70* 


.29 






23 


.60* 


.38 






24 


.67* 


,31 






25 


.31 


.67* 






26 


,20 


.78* 






27 


.80* 


.18 






28 


.40 


.57* 






29 


.63* 


.36 






30 


,11 


,88* 







O 

ERIC 
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Table B-3 

Initial Licensing Rules Test: Selection Proportions 

for Item Alternatives (Form #3) 



tam 


Alternative, 1 


Alternative 2 


Alternative 3 


Alternative 


i 


.71* 


.02 


.21 


.05 


2 


.04 


.12 


.64* 


.19 


3 


.08 


.20 


.68* 


.02 


4 


.69* 


.04 


.02 


.25 


5 


,02 


.67* 


,13 


.18 


6 


,67* 


.03 


,24 


.06 


7 


.22 


.10 


.04 


.62* 


8 


,04 


.81* 


.10 


,04 


9 


.68* 


.06 


.21 


.03 


10 


.93* 


.02 


.01 


.03 


11 


.14 


.09 


.10 


.65* 


12 


,12 


.12 


.05 


,70* 


13 


.08 


.05 


.07 


.79* 


14 


,10 


.01 


,01 


.86* 


u 


. XU 


,U4 




,u^ 


16 


.08 


.50* 


.36 


,02 


17 


.62* 


.07 


.17 


.13 


18 


,05 


,03 


,02 


.89* 


1? 


.22 


.43* 


.25 


.07 


20 


.08 


.05 


.82* 


,04 


21 


.60* 


.38 






22 


.84* 


.14 






23 


.69* 


,30 






24 


,80* 


.18 






25 


,75* 


,23 


/ 




26 


,14 


,84* 






27 


.29 


,69* 






28 


.91* 


.08 






29 


,84* 


,15 






30. 


.67* 


.32 









Cee 

1 

2 

4 

5 

6 

7 

8 

3 

10 

11 

12 

.13 

14 

i-O 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 
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Table B-4 

Initial Licensing Rules Test: Selection Proportions 

for Item Alternatives (Form #4) 



Alternative 1 Alternative 2 Alternative 3 Alternative A 



,01 


,01 


,97* 


,00 


,04 


.11 


. 74* 


.09 


,14 


.15 


.04 


,65* 


.02 


.04 


,85* 


.07 


, 68* 


.03 


.23 


,05 


,27 


,39 


.32 


,02 


,04 


,02 


.06 


, 87* 


,05 


,13 


,00 


,80* 


,02 


.04 


,06 


,87* 


,44* 


.16 


.18 


,21 


,60* 


,08 


.18 


,13 


,37* 


.13 


,35 


.14 


.74* 


.02 


.21 


,03 


.76* 


.03 


.19 


.01 




. J.U 


• iU 


, Oj" 


.70* 


,04 


.02 


.24 


.02 


,66* 


.13 


.17 


.36* 


. 22 


.08 


.33 


,02 


,91* 


.01 


.04 


.03 


.35 


.20 


,41* 


.37 


.61* 






,14 


.85* 






.86* 


,13 






.91* 


.08 






,30* 


.69 






,75 


.23* 






.08 


.91* 






.66* 


.22 






.88* 


,12 






.84* 


.15 
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Ite 

i 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 
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Table B-5 

Initial licensing Rules Test: Selection Proportions 

for Item Alternatives (Form # 5 ) 



Alternative 1 Alternative 2 Alternative 3 Alternative 4 



07 


,02 


.02 


,88* 


05 


,02 


.01 


,92* 


01 


,68* 


.12 


.18 


04 


,63* 


,14 


,18 


03 


,18 


.73* 


.06 


03 


,02 


,03 


.91* 


64* 


,04 


.23 


.08 


03 


,05 


.88* 


,03 


47* 


,13 


,01 


,38 


02 


,83* 


.10 


.04 


10 


,15 


.50* 


.24 


03 


.78* 


.16 


.01 


01 


.26 


.70* 


.02 


09 


.85* 


.03 


,03 


13 


,03 


.40 


4 3* 


06 


,10 


.08 


. 75* 


05 


.14 


.67* 


.11 


09 


.45* 


.35 


,08 


04 


.02 


.04 


.89* 


63* 


,12 


,15 


.09 


31 


,68* 






66* 


,32 






84* 


,15 






17 


,81* 






12 


,87* 






87* 


.12 


*** 




08 


.91* 






78* 


.20 






61* 


,38 






40 


.59* 
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Selection Proportions for Item Alternative 
Renewal Licensing Test Forms A to E 
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C-l 



Table C-l 

Renewal Licensing Rules Test: Selection Proportions 

for Item Alternatives (Form A) 



Item 


Alternative 1 


Alternative 2 


Alternative 3 


Alternative 4 


i 


,24 


.69 


.04 


.03 


2 


,46 


.43 


.02 


.08 


3 


,12 


,81 


.06 


,01 


4 


,18 


,15 


,64 


.02 


5 


.03 


.02 


,02 


.91 


6 


.02 


.90 


.04 


.03 


7 


,10 


,49 


.01 


,39 


8 


.62 


,12 


.00 


,24 


9 


,04 


.00 


.00 


,95 


10 


.02 


.02 


.02 


.93 


1 1 


Afl 


a n 


AH 


A r 


12 


.10 


.02 , 


.01 


.85 


13 


.03 


,02 


.01 


.94 


14 


.04 


,22 


,53 


,19 


IS 


,01 


,95 


.01 


.02 


16 


.01 


,02 


,97 


.01 


17 


.76 


,02 


,15 


.07 


18 


,05 


.79 


.12 


.03 


19 


,02 


.91 


.05 


.01 


20 


.01 


,94 


,02 


.03 
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Table C-2 

Renewal Licensing Rules Test; Selection Proportions 
for Item Alternatives (Form B) 



Item 


Alternative 1 


Alternative 2 


Alternative 3 


Alternative 4 


i 


.04 


,02 


,81 


,13 


2 


,09 


.86 


,02 


,02 


3 


.01 


.07 


.06 


c 85 


4 


,01 


.01 


,01 


,96 


5 


.84 


.03 


.12 


,01 


6 


.00 


,00 


.00 


■ .99 


7 


.46 


.43 


,02 


.07 


8 


,69 


.19 


,05 


,07 


9 


.08 


.02 


.02 


.88 


_L- U 


■ 




• U4 


,ui 


11 


.76 


.05 


.17 


.01 


12 


.12 


,02 


.01 


.84 


13 


.04 


.04 


.91 


.00 


14 


.50 


.10 


.26 


.13 


15 


,04 


,01 


.93 


.01 


16 


.16 


.79 


.01 


.03 


17 


,07 


.01 


,00 


,91 


18 


,71 


.02 


.00 


.26 


19 


.01 


.03 


.03 


.92 


20 


,04 


.03 


.01 


.92 
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1 1 e 

1 

2 

3 

4 

5 

6 

7 

8 

9 

xv 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
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Table "-3 

Renewal Licensing Rules Test: Selection Proportions 

for Item Alternatives (Form C) 



Alternative 1 Alternative 2 Alternative 3 Alternative 4 



04 


,02 


.80 


.14 


77 


.18 


.02 


.02 


06 


.80 


,01 


,11 


09 


.61 


.23 


,05 


16 


.17 


.12 


,53 


04 


.05 


.90 


.01 


83 


,02 


.09 


,05 


04 


.01 


.94 


.00 


01 


.00 


.00 


,99 


U1 


.03 


.U1 


,93 


03 


.02 


.03 


,94 


54 


.04 


.12 


,29 


01 


.87 


.08 


.03 


01 


.01 


,02 


.97 


07 


.03 


.01 


.88 


00 


.01 


.98 


.01 


10 


.75 


.03 


,11 


86 


.06 


.02 


,05 


03 


,08 


.10 


,78 


02 


.92 


.03 


,01 
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labile C-4 

Renewal Licensing Rules Lest: Selection Proportions 

for Item Alternatives (Form D) 



Item 


Alternative 1 


Alternative 2 


Alternative 3 


Alternative 4 


i 


.ii 


.79 


,08 


.01 


2 


,90 


.04 


,03 


,02 


3 


.06 


.91 


,03 


.00 


4 


.23 


.01 


,13 


,62 


5 


,01 


.00 


.01 


,99 


6 


,17 


.56 


,04 


,22 


7 


.13 


.02 


,83 


.02 


a 


.08 


.75 


,16 


.01 


9 


.90 


.00 


,05 


.03 


10 


.03 


.05 


. R6 


n6 


11 


,01 


.80 


,08 


'j 


12 


.00 


.01 


,97 




13 


.87 


.01 


.10 


. b. 


14 


.01 


.96 


.01 


.01 


15 


.05 


.81 


.01 


.12 


16 


,56 


.27 


,12 


.05 


17 


.01 


.02 


.00 


.96 


18 


.03 


.9i 


.03 


,02 


19 


.02 


.03 


.02 


,92 


20 


.59 


.11 


.01 


.29 
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Table C-5 

Renewal Licensing Rules Test: Selection Proportions 

for Item Alternatives (Form E) 



Item 


Alternative 1 


Alternative 2 


Alternative 3 


Alternative 4 


i 


,10 


.03 


.86 


.01 


2 


.03 


.95 


.01 


,00 


3 


.08 


.16 


,73 


.02 


4 


.82 


.01 


.09 


,06 


5 


.94 


.01 


.01 


.04 


6 


.00 


.00 


.00 


,99 


7 


.13 
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APPENDIX D 

Personal Data Form for Renewal License Applicants 





NORTH CAROLINA 
DEPARTMENT OP MOTOR VEHICLES 



Supplementary Information on Applicant for Renewal License 

Name L icense # 

Address _ 

Date of Birt h 

~~ (Month) (Day) (Year) 

Se x Race 

Circle grade in school applicant has completed: 1234 56? 89 10 11 12 

College: 1 2 3 4 5 6 

Occupation (Please describe in some detail) 



Marital Status: Never Marrie d ; Marrie d ;Divoroed or Separated j Widowed 

Did applicant have driver education course? ___ ___ 

Yes No 



If yes* wher e When completed_ 

(City) (State) 

How many years has applicant been driving? 

Make of ear applicant currently drives the most : 



Year of car applicant currently drives the most: 

How many miles 9 on the average 9 does applicant drive PER WEEK? 

Approximately how many miles did applicant drive in THE PAST YEAR? 

Score on written examination: Circle series: ABODE 

Score on oral examination: Circle series: F 1 2 3 4 5 



Was road test given? 



Passed:_ 

O 

ERIC 



Yes Wo 

Failed: Rules ; Signs_ 



Eyes 



Road 
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